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Abstract. In this dissertation we provide mathematical evidence that the concept of 
learning can be used to give a new and intuitive computational semantics of classical 
proofs in various fragments of Predicative Arithmetic. 

First, we extend Kreisel modified realizability to a classical fragment of first order 
Arithmetic, Heyting Arithmetic plus EMi (Excluded middle axiom restricted to for- 
mulas). We introduce a new realizability semantics we call "Interactive Learning-Based 
Realizability". Our realizers are self-correcting programs, which learn from their errors 
and evolve through time, thanks to their ability of perpetually questioning, testing and 
extending their knowledge. Remarkably, that capability is entirely due to classical princi- 
ples when they are applied on top of intuitionistic logic. 

Secondly, we extend the class of learning based realizers to a classical version PC J^ciass 
of VCJ- and, then, compare the resulting notion of realizability with Coquand game se- 
mantics and prove a full soundness and completeness result. In particular, we show there 
is a one-to-one correspondence between realizers and recursive winning strategies in the 
1-Backtracking version of Tarski games. 

Third, we provide a complete and fully detailed constructive analysis of learning as it 
arises in learning based realizability for HA + EMi, Avigad's update procedures and epsilon 
substitution method for Peano Arithmetic PA. We present new constructive techniques to 
bound the length of learning processes and we apply them to reprove - by means of our 
theory - the classic result of Godel that provably total functions of PA can be represented 
in Godel's system T. 

Last, we give an axiomatization of the kind of learning that is needed to computation- 
ally interpret Predicative classical second order Arithmetic. Our work is an extension of 
Avigad's and generalizes the concept of update procedure to the transfinite case. Trans- 
finite update procedures have to learn values of transfinite sequences of non computable 
functions in order to extract witnesses from classical proofs. 



A Margherita 



'Alors tous deux on est repartis 
Dans le tourbillon de la vie 
On a continue a tourner 
Tous les deux enlaces 
Tous les deux enlaces 
Tous les deux enlaces." 
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CHAPTER 1 



Introduction 

1.1. A Computational Semantics of Classical Proofs 

In this dissertation we provide mathematical evidence that the concept of learning can 
be used to give a new and intuitive computational semantics of classical proofs in various 
fragments of Predicative Arithmetic. The main definite result in this sense is a new realiz- 
ability semantics for HA + EMi, which we call "Interactive Learning Based Realizability" 
(shortly, learning based realizability). HA + EMi is first order intuitionistic Heyting Arith- 
metic with the principle of excluded middle over formulas, a classical axiom that eluded 
computational semantics for a long time and, probably, never had an intuitive one. 

EMi has already been extensively studied in terms of Curry-Howard correspondence 
(see [43]), that is, in terms of computational constructs that can be associated to it in 
order to extract computational information from classical proofs. Thanks to this approach, 
the constructive content of classical logic can be interestingly attained in terms of proof 
transformations, or reduction rules applied to the corresponding computational constructs. 
While these results are satisfying from the computational point of view, we feel they are not 
satisfying in terms of human understanding. Though this issue may at first appear marginal, 
it is not: classical proofs can now be "executed" thanks to Curry-Howard correspondence, 
but extracted programs are still very difficult to understand. Nowadays, there are research 
programs whose aim is precisely to understand programs extracted from classical proofs, 
since without this understanding programs cannot be analyzed, neither optimized nor im- 
proved, to begin with. Without high level grasp of a program, it is like having a black 
box, that in some mysterious way gives always correct answers. This phenomenon probably 
happens because classical principles are well understood only in terms of the computational 
devices they are associated to. Therefore, when one has to interpret a classical proof, he is 
forced to formalize it and then to extract the corresponding program. While this approach 
is feasible with small proofs, it is unmanageable with complex ones without great technical 
effort. This seems to explain why computational interpretations of classical logic are still 
not universally used by mathematicians and computer scientists. 

As for ourselves, we think that proof theory should offer a proof semantics: that is, 
not only a way of extracting the computational content of classical proofs, but also a high 
level explanation of what are the general ideas used by the programs extracted from proofs 
and what is the constructive meaning of quantifiers, logical connectives and axioms in 
the framework of classical logic. Only in this way, the general mathematician or computer 
scientist could extract intuitively the computational content hidden in some proof he wishes 
to analyze. Such a semantics was put forward a long time ago for intuitionistic logic - 
starting from Heyting semantics, passing through Kleene realizability to arrive to Kreisel 
modified realizability - but we think that in the case of classical logic there are still no 
ultimate results. 
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In this dissertation, we put a major effort to lay the ground for a proof semantics of 
Predicative Arithmetic based on the concept of learning. A learning based realizability in- 
terpretation of HA + EMi will be completely explained and formalized, compared with game 
semantics and thoroughly analyzed by constructive means. We will define a computational 
model of "intelligent" self- correcting programs, which learn from their errors and evolve 
through time, thanks to their ability of perpetually questioning, testing and extending their 
knowledge. Remarkably, that capability is entirely due to classical principles when they 
are applied on top of intuitionistic logic. We shall thus conclude that the computational 
content of classical fragment HA + EMi can be described in terms of learning. Moreover, 
we will introduce a more general concept of learning by levels, that generalizes Avigad's one 
[5] and will serve as a foundation for possible extensions of our learning based realizability 
to full first order Peano Arithmetic and even Predicative Analysis. The learning based 
computation interpretation of classical logic is a new and exciting field of research, but just 
started: in this thesis we give substantial contributions, but the path to follow is still long. 

In this chapter we give an overview of what are proof transformations, what are proofs 
semantics and what are the general ideas of learning in classical logic. We start with a 
short review of known results about intuitionistic logic, in order to explain how the issue 
"proof transformation vs. proof semantics" was solved in that case. We then explain the 
ideas behind epsilon substitution method, Coquand's game semantics and Avigad's update 
procedures, the three major sources of inspiration for our work and where the concept of 
learning first appeared, implicitly in the first case, explicitly and much more elegantly in the 
second and the third. These pioneering contributions are also the underpinnings of the more 
recent work of Berardi and de' Liguoro [11], which will be the starting point of this thesis. 
We conclude with a synopsis of the contribution and the structure of our dissertation. 

1.2. Proof Transformations and Proof Semantics in Intuitionistic Arithmetic 

Intuitionistic logic was the main method of reasoning used in mathematics until the 
eighteenth century. Existence of objects with special properties was mainly established 
by explicitly constructing objects with those properties. With Dedekind, Cantor, Hilbert, 
Brouwer, Frege and many others, non constructive reasoning became central, thanks to 
its great conceptual and simplifying power. Non constructive methods were immediately 
questioned by mathematicians such as Kronecker and others, but soon became generally 
employed and accepted. After the famous days of paradoxes in set theory and mathematics, 
however, a new season of concerns and foundational efforts began in mathematical logic. 
Brouwer, in particular, advocated the need for intuitionistic logic, which rejected some 
classical principles such as the excluded middle and impredicative definitions. 

Intuitionistic logic was born as a constructive logic, but what does that mean? Syntac- 
tically, it is presented just as a set of axioms and inference rules in a formal language. And 
the reason why, for example, excluded middle is left out remains inevitably obscure, if a 
semantics for assertions in intuitionistic logic is not provided; the excluded middle is a very 
natural and intuitive principle, after all, known in logic since Aristotle. 

The issue is informally solved through the so called Heyting semantics (see for example 
Girard [21]). According to Brouwer, in order to assert a statement, one has to provide some 
sort of construction. What is a construction? The following is Heyting's answer: 

(1) A construction of an atomic formula is a computation showing its truth. 
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(2) A construction of ^ A B is a pair formed by a construction of A and a construction 



(3) A construction of ^4 V is a pair whose first element is a boolean i such that: if 
i = True, then the second element of the pair is a construction of A, ii i = False, 
then the second element of the pair is a construction of B. 

(4) A construction of ^4 — > B is an algorithm taking as input a construction of A and 
returning as output a construction of B. 

(5) A construction oi\/x^A{x) is an algorithm taking as input a natural number n and 
returning as output a construction of A(n). 

(6) A construction of 3x^A{x) is a pair whose first element is a number n such that 
the the second element of the pair is a construction of A{n). 

Thanks to Heyting semantics, one immediately recognizes why the excluded middle A V 
^A is rejected by intuitionistic logic: a construction of AV^A would require to decide which 
one among A, -^A is true, which is not generally possible in algorithmic way. In Heyting 
semantics, each logical connective and quantifier is given a computational meaning, which 
is enough clear as to be intuitively used by humans with the aim of devising constructive 
proofs. 

1.2.1. Proof Transformations in HA. Let us consider the formal system HA, Heyt- 
ing Arithmetic, which is the intuitionistic version of the usual first order Peano Arithmetic 
PA. Since HA is born as a constructive logic, one expects it to be sound with respect to 
Heyting semantics. But since as a formal system HA does not mention at all the concept 
of Heyting construction, it is not obvious, given any provable formula, how to extract a 
construction from any of its proofs. 

The first method for implicitly extracting such a computational content is due to 
Gentzen [19], [20] and falls under the category of 'proof transformations. The idea is the 
following. One defines a set of transformation rules mapping proofs of a formula into proofs 
of the same formula. He then iterates these rules from the initial proof until he finds a 
proof with a special syntactical structure, and from this proof he obtains the constructive 
content "implicit" in the initial proof. We will not consider Gentzen technique (called "cut 
elimination") but instead a later one due to Prawitz, known as normalization, which applies 
to natural deduction proofs. We start from recalling the inference rules of HA. 



(1) A where A is a Peano axiom. 
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(4) 

(5) 

(6) 
(7) 



A B 

Ay B Ay B 
Ay B C C 
C 

^a^A A 
A[t/a] Va"A 
where t is a term in the language of HA 

A[t/a^] 3a^.A C 
^^7:^4" C 

A(0) ya^.A{a) A{S{a)) 



yoFA 



A natural deduction proof is a tree whose nodes are formulas obtained from their children by 
inference rules. Leaves of a deduction tree are either hypotheses or discharged hypotheses 
and all the standard management of discharged ones is assumed here (see for example van 
Dalen [15]). As usual, quantifier rules must be applied under the usual restrictions on the 
quantified variables. 

We are interested in a set of rules that allow to transform any proof of any formula A 
in a special normal form proof of A. We limit ourselves to the rules for implication and 
induction and we refer to [15] for a complete treatment. The proof transformations defined 
are the following. In any deduction, a sub-deduction 



B ^2 

A^ B A 
B 

is replaced by 

Vi[V2/A] 
B 

where Pi[I'2/^] is the deduction obtained from Di by replacing all the hypotheses A with 
the deduction 2?2 of A. Moreover, in any deduction, a sub-deduction 



Pi ^2 
^(0) Va".^(a) ^ A{S{a)) 

A{n) 

where n is a numeral, is replaced by a deduction 2?" of A{n), where by induction T)^ is 
defined as 

A{0) 

and for any numeral m, P^^"^) is defined as 
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V2 

yo^.A{a) A{S{a)) 
A{m) ^(S(m)) A{m) 
A{S{m)) 

It is possible to show that any natural deduction proof without hypotheses of a closed 
formula A can be transformed into a closed normal proof of A. Just by inspection and some 
straigthforward reasoning on the resulting proof shape, one can show that if ^ = 3x^B, 
then the last inference rule must be of the form 

Hence one automatically finds a witness t for 3x^B. li A = BiV B2, then the last inference 
rule must be of the form 

BoVBi 

with I € 0, 1: again a witness a required by Heyting semantics. The constructive information 
of the original proof of A can thus be found by normalization. 

In general, by normalization, it is possible to show that every formula provable in HA 
has a construction, in the sense of Heyting. There is a remark to be done: without Heyting 
semantics, the process of normalization would be perfectly non intelligible. One would only 
see a sequence of transformations performed on a initial proof until a normal form proof 
is found and, magically, a witness pops out of nothing. Moreover, normalization is not 
a technique that can be used in an intuitive way, because it requires full formalization of 
proofs and then to understand how the normalization process proceeds. 

1.2.2. Proof Semantics for HA. We now explain how a formal proof semantics for 
HA can be formulated in terms of Kreisel modified realizability [34]. Modified realizability 
is a formalization of Heyting semantics, carefully carved and designed for HA"^ (but we 
shall consider its restriction to HA). The idea is to restrict the class of algorithms used 
in Heyting's definition: only algorithm representable in Godel's system T (see chapter 2) 
are allowed. This is an important restriction: only bounded iteration is explicitly used by 
such algorithms and this property rules out the possibility that the computational content 
of proofs may be found by blind search and other constructively unjustified techniques (as 
it might happen with trivial Kleene-style realizers [31]). 

We start by associating types to formulas as to mirror the structure of the programs 
used in Heyting semantics. 

Definition 1.2.1 . (Types for realizers) For each arithmetical formula A we define a type 
1^1 of T by induction on A: 

(1) |P(ti,...,t„)| =N, 

(2) \AaB\ = \A\ X \B\, 

(3) \A\/ B\ = Bool X (1^1 X \B\), 

(4) \A^ B\ = \A\ \B\, 
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(5) \\/xA\ = N ^ 1^ 

(6) \3xA\ = N X 1^1 



We now define the Kreisel modified realizability relation t \\- C. It is clear that a 
modified realizer represents a construction in the sense of Heyting. 

Definition 1.2.2 (Modified Realizability). Assume t is a closed term of Godel's sys- 
tem T (see chapter 2), C is a closed formula, and t : |C|. Let i = ti, . . . , t„ : N. We define 
the relation t Ih C by induction and by cases according to the form of C: 

(1) t Ih P{i) if and only if P(t) = True 

(2) t\^ AaB if and only if tto* Ih A and vrit Ih B 

(3) t\\-AVB if and only if either pot = True and pit Ih A, or pot = False and p2t Ih B 

(4) t\\\- A^ B if and only if for all u, if u Ih A, then tu Ih^ B 

(5) t Ih \/xA if and only if for all numerals n, tn Ih ^[n/a;] 

(6) t Ih 3xA if and only for some numeral n, ttqI = n and nit Ih A[n/x\ 



Thanks to Kreisel modified realizability and to the correspondence between typed 
lambda terms and natural deduction proofs, one can give a new intuitive meaning to the 
normalization process for intuitionistic proofs. One does that by decorating inference rules 
with terms of system T in such a way that each of the natural deduction transformation 
rules we have previously seen correspond to a normalization step in the associated term. 
We shall present in chapter 3 the full decoration, here we consider only the case for — )-rules: 

u'r A^ B A tt h ^ 

wtVB \x\^\u h A^B 

We see that, finally, the idea of Heyting construction, in the form of modified realizability, 
is explicitly associated to the proof. Each inference rule either uses a realizer to compute 
something or defines a realizer. If we consider a decorated deduction T> 

u\- B V2 
Axl-^luh^^B thA 
[Xx\^\u)t h B 

the previously described proof transformation rule of T) in T)i\P2/A\ corresponds to the 
normalization step {\x^'^^u)t = u\t/x\, which is simply the evaluation of a function at its 
argument. 

The point to be made here is the following. First, one has a local interpretation of each 
inference rule in terms of realizability, that helps to understand what the rule itself means 
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and what it is going to accomplish from the computational point of view. Moreover, every 
axiom and every formula in general is given a computational meaning through realizabil- 
ity. In this way, one understands what arithmetical assertions mean from the constructive 
standpoint and what one has to do in order to constructively prove them. Without such 
a semantics, it would not be possible for a human to understand in a simple way what is 
the intuitive computational content that a proof offers. The goal of this dissertation is to 
extend this result to HA + EM i. 

1.3. Learning in Classical Arithmetic: A Brief History 

In this dissertation we provide a realizability semantics of HA + EMi classical proofs in 
terms of learning. The first question we have to answer is therefore the following: what is to 
be learned by realizers of classical proof si Surprisingly, the answer was anticipated a long 
time ago by Hilbert and his epsilon substitution method. 

1.3.1. Learning in the Epsilon Substitution Method. Kreisel's modified realizers 
have not enough computational power for deciding truth of formulas and thus they are 
prevented to realize classical principles such as the excluded middle; this limitation in turn 
prevents them to decorate classical proofs with the aim of finding witnesses for classically 
provable existential statements. However, as proven by Kreisel himself [33], one can extract 
from a classical proof in PA of any 112 formula yx^3y^ Pxy, with P decidable, a non trivial 
algorithm that given any number n finds a witness m such that Pnm is true. This result 
implies that even non constructive proofs of existence hide constructive information. 

Hubert's idea (for a modern account, see [5]) to circumvent the apparent inability of 
constructive methods to computationally interpret classical proofs, at least for n2 provable 
formulas, was to introduce non computable functions and to define through them classical 
witnesses. The role of those non computable functions, called epsilon substitutions, is to 
assign values to some non effectively evaluable epsilon terms. For each arithmetical formula 
A{x) one introduces a term exA{x), whose intended denotation can be any number n such 
that A{n) is true. Thus, exA{x) reads as "an x such that A{xy\ For every term t, one has 
therefore an axiom 

A{t) A{exA{x)) 

which captures completely the intended meaning of an epsilon term. Such an axiom is 
said to be a critical formula. Epsilon terms are not effectively computable, so one starts 
with an epsilon substitution S that gives to them dummy values. In order to extract 
the computational content of a classical proof, one has to satisfy all the critical formulas 
appearing in the proof. Fortunately, there is only a finite number of them in any proof and 
so nothing in principle makes the goal impossible. It turns out that one can learn values of 
epsilon terms by counterexamples. Suppose, for instance, that a critical formula 

A{t) A{exA{x)) 

is false under some substitution S. Then, if we denote with S{exA{x)) the value associated 
to exA{x) by S, we have that A{S{exA{x))) is false. However, if t evaluates to n under S, 
A{n) must be true! So, one updates the substitution S as to give to exA{x) the value n, 
because he has learned a witness through a counterexample. Things, however, are not so 
easy: if there are many critical formulas, this attempt of making true one of them, may 
make false another. Hilbert's Ansatz (approach) was to show that a specially defined series 
of this learning steps must terminate. 
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Hilbert's idea is brilliant, but it has never been used to give a semantics of classical 
proofs. As to ourselves, we shall use it and we can now anticipate that the goal of our 
learning based realizers will be to learn values of non computable functions. 

1.3.2. Learning in Coquand Game Semantics. The concept of learning in classical 
logic has been beautifully improved and reframed in terms of special Tarski games, in 
which the participating players are allowed to correct their moves: this is Coquand games 
semantics [14]. The interest, for our purposes, of this semantics, lies in the fact that the 
concept of recursive winning strategy in a standard Tarski game is nothing but a rephrasing 
of realizability: an adaptation of Tarski games to classical logic offers an opportunity to 
translate back the new intuitions in a realizability semantics, which is more suitable to proof 
interpretations. 

We first review what is a Tarski game (actually, the concept was explicitly defined 
by Hintikka (see [29]) and it is a sort of folklore). In a Tarski game there are two players, 
Eloise and Abelard, and a negation-and-implication-free formula B on the board (we assume 
that for every atomic formula its negation can also be expressed as an atomic formula). 
Intuitively, Eloise tries to show that the formula is true, while Abelard tries to show that 
it is false. Turns, sets of possible moves and winners are defined accordingly to the form of 
B: 

(1) If B = 3x^A{x), then Eloise has to choose a numeral n and B is replaced by the 
formula A{n). 

(2) If i? = \/x^A{x), then Abelard has to choose a numeral n and B is replaced by the 
formula A{n). 

(3) If i? = ^1 V A2, then Eloise has to choose a numeral i £ {0, 1} and B is replaced 
by the formula Ai. 

(4) U B = Ai A A2, then Abelard has to choose a numeral i € {0, 1} and B is replaced 
by the formula Ai. 

(5) If B is atomic and true, Eloise wins the game, otherwise Abelard wins. 

Informally, Eloise has a recursive winning strategy in this Tarski game, if she has an 
algorithm for choosing her next moves that enables her to win every play. It is possible 
to show that a formula has a construction in the sense of Heyting if and only if Eloise 
has a recursive winning strategy in its associated Tarski game. In this sense, Tarski games 
rephrase Heyting semantics. 

It is not surprising, then, that Eloise does not have a recursive winning strategy for every 
instance of the excluded middle EMi. Coquand solved this impasse by allowing Eloise to 
backtrack, i.e. to erase her moves and return to a previous state of the game (actually, 
when interpreting the cut rule, Coquand allowed also Abelard to backtrack, but we will not 
consider this case). Again, learning by counterexamples enters the scene. It is possible to 
prove that Eloise has a recursive winning strategy in the backtracking version of the Tarski 
game associated to EMi. Suppose, for example, that 

EMi := Vx". 3y^Pxy V ^y^^Pxy 
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and consider any play: we show how Eloise can win. Abelard has to move and, for some n, 
chooses the formula 

3y^Pny V ^y^^Pny 

Then it is the turn of Eloise, who believes that no witness for 3y^Pny can be found. So she 
chooses 

Again, Abelard for some m chooses 

-iPnm 

If Pnm is false, then Eloise wins. But if Pnin is true, she would have lost the standard 
Tarski game. However, according to the new rules, she can now backtrack to a previous 
position. Observe that Abelard has falsified Eloise's belief in the non existence of witnesses 
for the formula By^Pny, actually providing one witness. So Eloise backtracks to the position 

By^Pny V "iy^^Pny 

and this time chooses 
followed by 

Pnm 

and so she wins. 

We remark how close is this kind of learning to the one in epsilon substitution method. 
In both cases, some formula wished to be true is false. But from its falsehood one can 
always learn a new positive fact: a witness for an existential statement. 

1.3.3. Learning in Avigad's Update Procedures. In [5], Avigad has formulated 
an abstract axiomatization of learning as it implicitly appears in the epsilon substitution 
method for first order Peano Arithmetic. He has explicitly introduced the non computable 
functions needed by the epsilon method to elicit the computational content of classical 
proofs and formulated in a clear way the notion of update procedure that formalizes what 
we call "learning by levels". 

We give a definition slightly different from Avigad's, but analogous. Intuitively, a It- 
aly update procedure, with k G N"*", is a functional which takes as input a finite sequence 
/ = /i, . . . , /fc of functions approximating some oracles ct>i, . . . , 0^, such that each one of 
those functions is defined in terms of the previous ones. Then, it uses those functions to 
compute some witnesses for some provable formula of PA. Afterwards, it checks whether 
the result of its computation is sound. If it is not, it identifies some wrong value fi{n) used 
in the computation and corrects it with a new one. 

Definition 1.3.1 (Update Procedures). A k-ary update procedure, k e N+, is a con- 
tinuous function lA : {H ^ N)*^ — ?> U {0} (i.e., its output is determined by a finite number 
of values of the input functions) such that the following holds: 

(1) for all function sequences f = fi, ■ ■ ■ , fk 

hif = {i,n,m) =^ 1 <i < k 

(2) for all function sequences f = fi, ■ ■ ■ , fk and g = gi, gk, ior all 1 < i < k, if 



i) for aU j < i, fj = g^- 
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ii) Uf = {i, n, m), gi{n) = m and Ug = {i, h, I) 
then h ^ n. 

If is a k-avy update procedure, a zero for U is a sequence / = /i, • • • , /fc of functions such 
that Uf = 0. 

Condition (2) of definition 1.3.1 means that the values of the i-th function depend on 
the values of some of the functions fj, with j < i, and learning on level i is possible only 
if all the lower levels j have "stabilized". In particular, if Z// is a A;-ary update procedure 
and / : (N — 7> N)'' is a sequence of functions approximating the oracles 0i, . . . there 
are two possibilities: either / is a fine approximation and then Uf = 0; or / is not and 
then Uf = {i,n,m), for some numerals n, m: U says the function fi should be updated 
as to output m on input n. Moreover, if Uf = {i,n,m), one in a sense has learned that 
= m: by definition of update procedure, if is a function sequence agreeing with / in 
its first i — 1 elements, gi is another candidate approximation of <t>i and gi{n) = m, then Ug 
does not represent a request to modify the value of gi at point n, for Ug = (i, h, I) implies 
h ^ n. 

The main theorem about update procedures is that they always have zeros and these 
latter can be computed through learning processes guided by the former. Intuitively a zero of 
an update procedure represents a good approximation of the oracles used in a computation, 
and in particular a good enough one to yield some sought classical witness. 

1.4. Learning in Classical Arithmetic: Contributions and Structure of This 

Dissertation 

The aim of this dissertation is to study and describe the computational content of 
classical proofs in terms of learning. In particular, the contributions and the structure of 
this dissertation are the following. 

1.4.1. Chapter 3. Our first contribution is to put together in a novel way the ideas 
contained in the epsilon substitution method, Coquand game semantics and Avigad's update 
procedures in order to define a realizability semantics of proofs which extends in a simple way 
Kreisel modified realizability to the classical system HA + EMi: we shall call it (interactive) 
learning based realizability. 

In a few words, learning based realizability describes a way of making oracle compu- 
tations more effective, through the use of approximations of oracle values and learning of 
new values by counterexamples. A learning based realizer is in the first place a term of 
system Tciass) which is a simple extension Godel's system T plus some oracles of the same 
Turing degree of an oracle for the Halting problem. Of course, if a realizer was only this, 
it would be ineffective and hence useless. Therefore, learning based realizers are computed 
with respect to approximations of the oracles of Tciass and thus effectiveness is recovered. 
Since approximations may be sometimes inadequate, results of computations may be wrong. 
But a learning based realizer is also a self- correcting program, able to spot incorrect oracle 
values used during computations and to correct them with right values. The new values 
are learned, surprisingly, by realizers of EMi and all the oracle values needed during each 
particular computation are acquired through learning processes. Here is the fundamental 
insight: classical principle may be computationally interpreted as learning devices. 



1.4. LEARNING IN CLASSICAL ARITHMETIC: CONTRIBUTIONS AND STRUCTURE OF THIS DISSERTATION 



Our realizability semantics allows not only to extract realizers as usual by decorating 
classical proofs, but also to understand the intuitive meaning, behaviour and goals of the 
extracted realizers. 

1.4.2. Chapter 4. Our second contribution is, first, to extend the class of learning 
based realizers to a classical version VCJ-quss of VCJ- and, then, to compare the result- 
ing notion of realizability with Coquand game semantics and prove a full soundness and 
completeness result. In particular, we show there is a one-to-one correspondence between 
realizers and recursive winning strategies in the 1-Backtracking version of Tarski games. 

The soundness theorem should be useful to understand the significance and see possible 
uses of learning based realizability. The idea is that playing games represents a way of 
challenging realizers and of seeing how they react to the challenge by learning from failure 
and counterexamples. 

The proof of the completeness theorem in our view, moreover, has an interesting feature. 
In a sense, it is the first application of the ideas of learning based realizability to a concrete 
non trivial classical proof, which is our version of the one given by Berardi et al. [9] . That 
proof classically shows that if Eloise has recursive winning strategy in the 1-Backtracking 
Tarski game associated to a formula A, then she also has a winning strategy in the Tarski 
game associated to A (but a non computable strategy, only recursive in an oracle for the 
Halting problem). We manage to associate a constructive content to this seemingly ineffec- 
tive proof and find out that it hides a learning mechanism to gain correct oracle values from 
failures and counterexamples. We then transform this learning mechanism into a learning 
based realizer. 

1.4.3. Chapter 5. Our third contribution is a complete and fully detailed constructive 
analysis of learning as it arises in learning based realizability for HA + EMi, Avigad's update 
procedures and epsilon substitution method for Peano Arithmetic PA. We present new 
constructive techniques to bound the length of learning processes and we apply them to 
reprove - by means of our theory - the classic result of Kreisel that provably total functions 
of PA can be represented in Godel's system T. An interesting novelty is that we develop type- 
theoretic techniques to reason and prove in a new way theorems about update procedures 
and epsilon substitution method. A notable byproduct of our work is the introduction of 
a "constructive" non standard model of Godel's system T. Our analysis is also a first step 
toward an extension of learning based realizability to full PA. 

1.4.4. Chapter 6. Our last contribution is an axiomatization of the kind of learning 
that is needed to computationally interpret Predicative classical second order Arithmetic. 
Our work is an extension of Avigad's and generalizes the concept of update procedure to the 
transfinite case. Transfinite update procedures have to learn values of transfinite sequences 
of non computable functions in order to extract witnesses from classical proofs. We shall 
present several proofs of the fact that transfinite update procedures have zeros. The last 
one uses methods of type theory and in particular bar recursion: the algorithms presented 
are powerful and yet quite simple. The interest of our results is twofold. First, we extend 
Avigad's intuitive description of the learning content of the epsilon substitution method to 
the second order case. Secondly, we take a first step toward the extension of learning based 
realizability to Predicative second order Arithmetic, since we have isolated the concept of 
learning that will have to be employed. 



CHAPTER 2 



Technical Preliminaries 

2.1. Background 

This dissertation is almost self-contained from the technical point of view. We cover the 
needed background here by reviewing what will be our main technical tool: Godel's System 
T. 

2.1.1. Godel's system T. Godel's system T (see [21], for example) is simply typed 
A-calculus, enriched with natural numbers, booleans, conditional if-r and primitive recursion 
Rt in all types together with their associated reduction rules. We start by defining the types 
of system T. 

Definition 2.1.1 (Types of System T). The set of types of system T is defined induc- 
tively as follows: 

(1) N and Bool are types. 

(2) If U, V are types, U xV, U are types. 

The type N represents the set N of natural numbers and Bool represents the set B = 
{True, False} of booleans, while product types T x U and arrows types T ^ U represent 
respectively cartesian products and function spaces. We assume — > associates to the left: 

T^U^V = T^{U^V) 

Definition 2.1.2 (Terms of System T). We define the terms of system T inductively 
as follows: 

(1) For all types U, the variables Xq, . . . , x^, . . . are terms of type U. 

(2) is a term of type N. True and False are terms of type Bool. 

(3) For every type T, 'ifjj is a term (constant) of type U := Bool —^T—^T^T. Terms 
of the form '\fTt1t2t3 will be written in the more legible form if ti then t2 else t^. 

(4) For every type T, Rfj is a term (recursion constant) of type ?7 := T — )■ (N — > (T — >■ 
T)) ^ N ^ T. The type T in R^^ will be omitted whenever inferable from the 
context. 

(5) If t is of type N, then S(t) is a term of type N. 
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(6) If t and u are terms of types respectively U ^ V and U, then tu is a term of type . 

(7) If t is a term of type U x V, then not and vrif are terms of types respectively U 
and V. 

(8) If u and v are terms of types respectively U and V, then (u, is a term of type 

(9) If is a term of type V and a variable, then Xx^v is a term of type U ^ V. 

As usual, given two terms u and t and a variable with n[t/a;] we shall denote the 
term resulting from u by replacing all free occurrences of x in n with t, avoiding capture of 
variables. 

We now give the reduction rules that explain the computational meaning of the syntax 
of system T. 

Definition 2.1.3 (One Step /3o Reduction and /3 Reduction). We define a binary 
relation /3o between terms of system T as the least relation satisfying the following properties: 

(1) If u u' and v Pq v' , then for all terms u,v, it holds that uv (Sq u'v, uv /3q uv', 
{u,v) (3o {u',v) and {u,v) /3o {u,v'). 

(2) If u /3o u', then Xx'^u Pq Xx^u' and vrjU Pq ttiu' for i = 0, 1. 

(3) {Xx^u)t Po u[t/x]. 

(4) TTi{uo,ui) Po Ui for i = 0, 1. 

(5) RuvQ Po u. 

(6) RuvS{t) Po vt{Ruvt). 

(7) if True then ui else U2 Po ui and if False then ui else U2 Po U2- 

(8) t P t' \i t = t' ov t Po ui Po U2 Po ■ ■ ■ Po Un Po t' for some terms ui,U2 ■ ■ ■ , Un- 

(9) We say that t is in normal form if t Po t' does not hold, for every t' . 

We now define the equality rules for terms of system T. Throughout the dissertation 
we will write u = t if the equation is provable by means of the following rules. 

Definition 2.1.4 (Equational Theory of T). We hst the axioms of equality for system 
T: 

(1) t = t. 
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(2) 



u, then u = t. 



(3) 



If i 



u and u = V, then t = v. 



(4) 



If M 



U' 



•! and V = v' , then txz; = u'v' and (ti, = (n', u'). 



(5) 



If M 



u', then Ax^n = Ax^ti' and -KiU = ttiu' for i = 0, 1. 



(6) {\x^u)t = u[t/x]. 

(7) 7rj(uo, ui) = for i = 0, 1. 

(8) RuvQ = u. 

(9) RiiwS(i) = vt{Ruvt). 

(10) if True then ui else n2 = ui and if False then ui else U2 = U2- 



Every term of Godel's system T has a unique normal form (see [21]). 

Theorem 2.1.1 (Normalization and Church-Rosser Property). For every term t 
of system T there exists n € N such that t (3q ti /3o . . . /3o tm implies m < n. Hence, t has a 
normal form. 

Moreover, if t /3 ti and t f3 t2, then there exists t' such that ti f3 t' and t2 (3 t' . As 
consequence, t has a unique normal form. 

A term is closed if it has no free variables; a numeral is a term of the form S'^(O), with 
n G N, having inductively defined S°(0) := and S"+^(0) := S(S"(0)). We will constantly 
use the following characterization of normal forms (see again [21]). 

Theorem 2.1.2 (Normal Form Characterization for System T). Assume A is an 
atomic type. Then any closed normal term tofTof type A is either a numeral n : N or a 
boolean True, False : Bool. 
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Interactive Learning-Based Realizability for Heyting 

Arithmetic with EMi 

Abstract. In this chapter, we extend Kreisel modified realizability to a classical fi-agment 
of first order Arithmetic, Heyting Arithmetic plus EMi (Excluded middle axiom restricted 
to formulas). In particular, we introduce a new realizability semantics we call "Interac- 
tive Learning-Based Realizability". Our realizers are self- correcting programs, which are 
able to learn something from every failure and use the new knowledge to correct them- 
selves. We build our semantics over Avigad's fixed point result [5], but the same semantics 
may be defined over different constructive interpretations of classical arithmetic (in [10], 
continuations are used). Our notion of realizability extends intuitionistic realizability and 
differs from it only in the atomic case: we interpret atomic realizers as "learning agents" . 

3.1. Introduction 

From now on, we will call HA Heyting Intuitionistic Arithmetic, with a language includ- 
ing one symbol for each primitive recursive predicate or function (see [15] or section 3.3). 
We call Sj'-formulas the set of all formulas 3x.P{x, y) for some primitive recursive predicate 
P, and EMi the Excluded middle axiom restricted to T,^-formulas. For a detailed study of 
the intuitionistic consequences of the sub-classical axiom EMi we refer to [1]. 

This chapter is based on Aschieri and Berardi [3]. We extend Berardi and de' Liguoro 
([7], [10]) notion of atomic realizability - originally conceived for quantifier free primitive 
recursive Arithmetic plus EMi - to full predicate logic, namely Heyting Arithmetic with 
EMi (HA-|-EMi). Our idea is to interpret classical proofs as constructive proofs on a 
suitable structure Af for natural numbers and maps of Godel's system T, by applying to 
the semantics of Arithmetic the idea of "finite approximation" used to interpret Herbrand's 
Theorem. We extend intuitionistic realizability to a new notion of realizability, which we call 
"Interactive learning-based Realizability" . We provide a term assignment for the standard 
natural deduction system of HA -|- EMi, which is surprisingly equal in all respects to that 
of HA, but for the fact that we have non-trivial realizers for atomic formulas and a new 
realizer for EMi. 

Our semantics is "local": we do not introduce a global variable representing an "ex- 
ternal" goal, different for each particular proof one wants to interpret, as in continuation 
interpretation, in Friedman's A-translation and in Krivine's Classical Realizability. The 
goal of realizers is fixed, "internal", and is either to provide right constructions or to learn 
new information about excluded middle. In this way, we interpret EMi and thus classical 
proofs locally and step-by-step, in order to solve a major problem of all computational 
interpretations: global illegibility, which means that, even for simple classical proofs, it is 
extremely difficult to understand how each step of the extracted program is related to the 
ideas of the proof, and what it is the particular task performed by each subprogram of the 
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extracted program. The main sources of inspiration of this chapter are works of Kleene, 
Hilbert, Coquand, Hayashi, Berardi and de' Liguoro and Avigad. 

Intuitionistic Realizahility revisited. Recall chapter 1. In [31], Kleene introduced the 
notion of realizahility, a formal semantics for intuitionistic arithmetic. Later, Kreisel [34] 
defined modified realizahility, the same notion but with respect to Godel's system T in- 
stead of Kleene's formalism of partial recursive functions. Realizahility is nothing but a 
formal version of Heyting semantics for intuitionistic logic, translated into the language of 
arithmetic. 

Intuitively, realizing a closed arithmetical formula A means exhibiting a computer pro- 
gram - called realizer - able to calculate all relevant information about the truth of A. 
Hence, realizing a formula Ay B means realizing A or realizing after calculating which 
one of the two is actually realized; realizing a formula 3xA{x) means computing a numeral 
n - called a witness - and realizing A{n). 

These two cases are indeed the only ones in which we have relevant information to 
calculate about the truth of the corresponding formula, and there is a decision to be made: 
realizing a formula \/xA means exhibiting an algorithm which takes as input a numeral n 
and gives as output realizers of A{n)] realizing a formula A f\ B means realizing A and 
realizing B\ realizing A ^ B means providing an algorithm which takes as input realizers 
of A and gives realizers of i?; in these cases we provide no information about the formula 
we realize and we only take the inputs we will use for realizing existential or disjunctive 
formulas. Finally, realizing an atomic formula means that the formula is true: in this case, 
the realizer does nothing at all. 

Hence, intuitionistic realizahility closely follows Tarski's definition of truth - the only 
difference being effectiveness: for instance, while Tarski, to assert that 3xA is true, con- 
tented himself to know that there exists some n such that A{n) is true, Kleene asked for a 
program that calculates an n such that A{n) is true. 

Intuitionistic natural deduction rules are perfectly suited to preserve realizahility. In 
order to actually build realizers from intuitionistic natural deductions, it suffices to give 
realizers for the axioms. Since our goal is to interpret classical connectives using Heyting 
and Kleene interpretation of intuitionistic connectives, then a first, quite naive idea would 
be the following: if we devised realizers for Excluded Middle, we would be able to extend 
realizahility to all classical arithmetic. 

Unfortunately, from the work of Turing it is well known that not every instance of 
Excluded Middle is realizable. If Txyz is Kleene's predicate, realizing \/xyy.3zTxyz V 
\/z^Txyz implies exhibiting an algorithm which for every n, m calculates whether or not 
the n-th Turing machine halts on input m: the halting problem would be decidable. Hence, 
there is no hope of computing with effective programs all the information about the truth 
of Excluded Middle. 

However, not all is lost. A key observation is the following. Suppose we had a realizer O 
of the Excluded Middle and we made a natural deduction of a formula 3xA actually using 
Excluded Middle; then, we would be able to extract from the proof a program containing 
O as subprogram, able to compute the witness for 3xA. Given the effectiveness of after a 
finite number of steps - and more importantly, after a finite number of calls to O - n would 
yield the required witness. It is thus clear that u, to perform the calculation, would use 
only a finite piece of information about the Excluded Middle. This fundamental fact gives 
us hope: maybe there is not always necessity of fully realizing Excluded Middle, since in 
finite computations only a finite amount of information is used. If we were able to gain 
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that finite information during the computation, as it is the case in the proof of Herbrand's 
Theorem, we could adapt intuitionistic realizabihty to Classical Logic. 

Herbrand's Theorem and the idea of "finite approximation" . (A corollary of) Herbrand's 
Theorem says that if a universal first order theory T, in a suitable language supporting 
definition by cases, proves a statement 3xP{x), then one can extract from any proof a term 
t and closed instances Ai, . . . , An of some universal formulas of T such that AiA...A An — )• 
P{t) is a propositional tautology. So, even using classical logic, one can define witnesses. 
The problem is that the functions occurring in t may not be computable, because the 
language of T is allowed to contain arbitrary functions. However, given the finiteness of the 
information needed about any function used during any finite computation of t, in order 
to carry out actual calculations one would only have to find finite approximations of the 
non- computable functions involved, thus recovering effectiveness. We choose to follow this 
intuition: we will add non-computable functions to our language for realizers and exploit 
the existence of these ideal objects in order to find concrete computational solutions. 

This general idea dates back to Hilbert's e-substitution method (for a neat reformulation 
of the e- method see for example Avigad [5] ) . As noted by Ackermann [2] , the e- substitution 
method may be used to compute witnesses of provable existential statements of first order 
Peano Arithmetic. The procedure is simple: introduce Skolem functions (equivalently, e- 
terms) and correspondent quantifier free Skolem axioms in order to reduce any axiom to 
a quantifier free form; take a P^-proof of a sentence 3xP{x) and translate it into a proof 
using as axioms only universal formulas; then apply Herbrand's theorem to the resulting 
proof, obtaining a quantifier free proof of P{t), for some term t of the extended language; 
finally, calculate a suitable finite approximation of the Skolem functions occurring in t and 
calculate from t an n such that P{n) holds. 

However, while proofs in quantifier free style are very simple combinatorial objects, they 
lose the intuitive appeal, the general concepts, the structure of high level proofs. Hence, it 
may be an impossible task to understand extracted programs. Moreover we have a com- 
putational syntactic method but no semantics of proofs and logical operators based on the 
idea of "finite approximation" , as the realizability interpretations are based on the idea of 
"construction". However, in the e-method, albeit only for quantifier free formulas, we see in 
action the method of intelligent learning, driven by the Skolem axioms used in the proofs. 
One of the aims of this chapter is to extend this "semantics of learning" from atomic propo- 
sitions to individuals, maps, logical connectives and quantifiers of full natural deduction 
proofs. An important contribution comes from Coquand [14]. 

Coquand's Game Semantics for Classical Arithmetic. Computing all relevant infor- 
mation about the truth of a given formula A is not always possible. In [14] and in the 
context of game semantics, Coquand introduced a new key idea around this problem: the 
correspondence between backtracking (in game theory, retracting a move) and "learning", 
a refinement of the idea of "finite approximation". If we cannot compute all the right in- 
formation about the truth of a formula, maybe we could do this if we were allowed to make 
finitely many mistakes and to learn from them. 

Suppose, for instance, we have the formula \lx3yPxy V \/y^Pxy, but we have no algo- 
rithm which, for all numeral n given as input, outputs false if \/y^Pny holds and outputs 
true if 3yPny holds. Then we may describe a learning algorithm r as follows. Initially, 
for all n given as input, r outputs false. Intuitively, r is initially persuaded - following the 
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principle "if I don't see, I do not believe" - that for all numeral n there is no numeral m such 
that Pnm holds. Hence, when asked for his opinion about the formula 3yPny V Vy^Pny, 
r always says: 3yPny is false. However, if someone - an opponent of r - to show that r 
is wrong, comes out with an m such that Pnm holds, r realizes indeed to be mistaken, 
and stores the information ^^Pnm is true". Then, the next time being asked for an opinion 
about 3yPny V \/y^Pny, r will say: true. In other words, such r, after at most one "mind 
changing" , would be able to learn the correct answer to any question of the form: "which 
one among 3yPny, \/y^Pny does hold?". This is actually learning by counterexamples and 
is the key idea behind Coquand's semantics. 

Our question is now: can we formulate a realizability notion based on learning by 
counterexamples in order to extend Kreisel's interpretation to all individuals, maps and 
connectives of the sub-classical Arithmetic HA + EMi? Following Hayashi [28], in our solu- 
tion we modify the notion of individual, in such a way that individuals change with time, 
and realizers "interact" with them. 

Hayashi' s Proof Animation and Realizability. In [28], Hayashi explains a notion of 
realizability for a sub-classical arithmetic, called limit computable mathematics. Basing 
his analysis on ideas of Gold [22], he defines a Kleene's style notion of realizability equal 
to the original one but for the fact that the notion of individual changes: the witnesses of 
existential and disjunctive formulas are calculated by a stream of guesses and "learned in 
the limit" (in the sense that the limit of the stream is a correct witness). An individual a is 
therefore a computable map a : N — )> N, with a{t) representing the value of the individual 
at time t. 

For instance, how would Hayashi realize the formula \lx.3yPxy V \/y^Pxy? He would 
define an algorithm H as follows. Given any numeral n, H would calculate the truth 
value of My < nPny. Then the correct answer to the question: "which one among 3yPny, 
My^Pny does hold?" is learned in the limit by computing P(n,0), 1), P(n, 2),..., 
P(n, fc),. . . and thus producing a stream of guesses either of the form false, false, false,. . . , 
true, true,. . . , true,. . . or of the form false, false, false, . . . , false, . . . , the first stabilizing in 
the limit to true, the second to false. Hayashi's idea is to perform a completely blind and 
exhaustive search: in such a way, the correct answer is guaranteed to be eventually learned 
(classically). Hayashi's realizers do not learn in an efficient way: in Hayashi's notion of 
realizability the only learning device is to look through all possible cases. Instead, we want 
to combine the idea of individual as limit, taken from Hayashi, with notion of learning in 
which the stream of guesses is driven by the proof itself as in Coquand's game semantics. 
For the quantifier-free fragment, this was done by Berardi [7] and Berardi-de' Liguoro [10]. 

Realizability Based on Learning: Berardi-de' Liguoro interpretation. We explain [10] 
using Popper's ideas [40] as a metaphor. According to Popper, a scientific theory relies 
on a set of unproved - and unprovable - hypotheses and, through logic, makes predictions 
susceptible of being falsified by experiments. If a prediction is falsified, some hypothesis 
is incorrect. In front of a counterexample to a theory's prediction, one must modify the 
set of hypotheses and build a better theory, which will be tested by experiments, and so 
on. Laws of Nature are universal statements, that cannot be verified, but are suitable to 
falsification. We may explain the link between falsifiable hypotheses and EMi. For every 
n, given an instance 3y.Pny V My.^Pny of EMi (with P atomic), we may formulate an 
hypothesis about which side of the disjunction is true. If we know that Pnm is true for 
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some m, we know that 3y.Pny is true. Otherwise we may assume My.^Pny as hypothesis, 
because it is a falsifiable hypothesis. 

We formahze the process of making hypotheses about EMi by a finite state of knowledge, 
cahed S, cohecting the instances Pnm which we know to hold, e.g. by direct calculation. If 
we have evidence that Pnm holds for some m (that is, Pnm G S) we know that 3yPny is 
true; in the other case, we assume that My^Pny is true. So S defines a set of hypotheses on 
EMi, of the form My^Pny: universal falsifiable statements. Using S a realizer r may effec- 
tively decide which side of a given instance of EMi is true, at the price of making mistakes: 
to decide if My^Pny is true, r looks for any Pnm in the finite state S and outputs "false" 
if the research is successful, "true" otherwise. If and when from an hypothesis My^Pny we 
obtain some false conclusion ^Pnm, the realizer r returns the additional knowledge: ^^Pnm 
is true" , to be added to S. 

Extending Berardi-de' Liguoro interpretation to HA + EMi. In our chapter, we interpret 
each classical proof p oi A in HA + EMi by a "learning realizer" r. r returns a "prediction" 
of the truth of this formula, based on the information in S, and some additional knowledge 
in the case the prediction is effectively falsified. For example, in front of a formula 3x.AAB, 
a realizer r predicts that A{n) A B{n) is true for some numeral n (and since n depends on 
S, in our model we change the notion of individual, interpreting "numbers" as computable 
maps from the set of bases of knowledge to N). Then r predicts, say, that B{n) is true, 
and so on, until r arrives at some atomic formula, say ^Pnm. Either Pnm is actually true, 
or r is able to effectively find one or more flawed hypothesis Vx.-iQinix, . . . ^Mx.^Q^n^^x 
among the hypotheses used to predict that Pnm is true, and for each flawed hypothesis 
one counterexample Qinimi, . . . ^Qk^k^nt- In this case, r requires to enlarge our state of 
knowledge S by including the information "Qinimi is true", . . . , ^^Qk^k^^k is true". 

Our Interactive Realizability differs from Intuitionistic Realizability in the notion of 
individual (the value of an individual may depend on our knowledge state), and in the 
realizability relation for the atomic case. In our interpretation, to realize an atomic formula 
does not mean that the formula is true, but that the realizer requires to extend our state of 
knowledge S if the formula is not true. The realizer is thought as a learning device. Each 
extension of S may change the value of the individuals which are parameters of the atomic 
formula, and therefore may make the atomic formula false again. Then the realizer requires 
to extend S again, and so forth. The convergence of this "interaction" between a realizer 
and a group of individuals follows by Avigad's fixed point thm. [5] (a constructive proof 
may be found in [7]), and it is the analogue of the termination of Hilbert's e-substitution 
method. 

Why the Arithmetic HA + EMi instead of considering the full Peano Arithmetic? We 
have two main reasons. First, we observe that EMi enjoys a very good property: the 
information about its truth can be computed in the limit, in the sense of Gold [22], as 
we saw en passant when discussing Hayashi's realizability. This implies that witnesses for 
existential and disjunctive statements too can be learned in the limit, as shown in Hayashi 
[28]. In chapter 4 we show that realizers which we will be able to extract from proofs have a 
straightforward interpretation as winning strategies in 1-Backtracking games [9], which are 
the most natural and simple instances of Coquand's style games. Secondly, a great deal of 
mathematical theorems are proved by using EMi alone ([1], [8]). Third, as shown in chapter 
5, already HA + EMi - plus Godel's double negation translation - suffices to interpret all 
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provably total recursive functions of PA, with the advantage of ehminating the extra step 
of Friedman or Dialectica translation (see Kohlenbach for [32] these latter) 

Plan of the Chapter. The chapter is organized as follows. In §3.2 we define the term 
calculus in which our realizers will be written: a version of Godel's system T, extended 
with some syntactic sugar, in order to represent bases of knowledge (which we shall call 
states) and to manipulate them. Then we prove a convergence property for this calculus 
(as in Avigad [5] or in [7]). In §3.3, we introduce the notion of realizability and prove our 
Main Theorem, the Adequacy Theorem: "if a closed arithmetical formula is provable in 
HA + EMi, then it is realizable". In §3.4 we conclude the discussion about our notion of 
realizability by comparing it with other notions of realizability for classical logic, then we 
consider some possible future work. 

3.2. The Term Calculus 

In this section we formalize the intuition of "learning realizer" we discussed in the 
introduction. 

We associate to any instance 3yPxy y My^Pxy of EMi (Excluded Middle restricted to 
S^-formulas) two functions XP ^tnd '.pp. The function XP takes a knowledge state 5, a 
numeral n, and returns a guess for the truth value of 3y.Pny. When this guess is "true" 
the function Lfp returns a witness m of 3y.Pny. The guess for the truth value of 3y.Pny 
is computed w.r.t. the knowledge state S, and it may be wrong. For each constant s 
denoting some knowledge state S, the function Ax" xpis,x) is some "approximation" of an 
ideal map Xp(x), the oracle returning the truth value of 3y.Pxy. In the same way, 
the function Ax" (j)p{s,x) is some "approximation" of an ideal map Ax" <t>p(x), the Skolem 
map for 3y.Pxy, returning some y such that Pxy if any, and otherwise. The Skolem 
axioms effectively used by a given proof take the place of a set of experiments testing the 
correctness of the predictions made by ipp{s,x),xp{s,x) about Xp(x), <t>p(x) (we do not 
check the correctness of ipp,xp in an exhaustive way, but only on the values required by 
the Skolem axioms used by a proof). 

Our Term Calculus is an extension of Godel's system T (see chapter 2 or [21]). From 
now on, if t, u are terms of T with t = u we denote provable equality in T. If A: € N, the 
numeral denoting k is the closed normal term k = S'^(O) of type N. All closed normal terms 
of type N are numerals (see chapter 2). We recall that any closed normal term of type Bool 
in T is True or False. 

We introduce a notation for ternary projections: if T = ^ x (B x C), with Po,Pi,P2 we 
respectively denote the terms ttq. Ax : T.7ro(7ri(x)), Ax : T.7ri(7ri(x)). If n = (uq, {ui,U2)) '■ 
T, then piU = tij in T for i = 0,1, 2. We abbreviate {uq, {ui,U2)) '■ T with {uq, ui, U2) ■ T. 
We formalize the idea of "finite information about EMi" by the notion of state of knowledge. 

Definition 3.2.1 (States of Knowledge and Consistent Union). We define: 

(1) A A;-ary predicate of T is any closed normal term P : n'^ — ?• Bool of T. 

(2) An atom is any triple {P, ft, m), where P is a (A; + l)-ary predicate, ft, m are k + \ 
numerals, and Pnm = True in T. 

(3) Two atoms {P, ft, m), {P' , fi' , m') are consistent P = P' and n = ft' imply m = m! . 
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(4) A state of knowledge, shortly a state, is any finite set S of pairwise consistent atoms. 

(5) Two states 5i, 5*2 are consistent if Si U ^2 is a state. 

(6) S is the set of all states of knowledge. 

(7) The consistent union S1US2 oi 81,82 € S is 5i U ^2 € S minus all atoms of ^2 
which are inconsistent with some atom of 5i. 



We think of an atom {P,n,m) as the code of a witness for 3y.P{n,y). Consistency 
condition allows at most one witness for each 3y.P{n, y) in each knowledge state 8. Two 
states 5*1 , 82 are consistent if and only if each atom of 81 is consistent with each atom of 
S2. 

81IA82 is an non-commutative operation: whenever an atom of 81 and an atom of 5*2 
are inconsistent, we arbitrarily keep the atom of 81 and we reject the atom of 5*2, therefore 
for some <S'i,52 we have 81U82 7^ 82U81. U \s a. "learning strategy", a way of selecting a 
consistent subset of 81 U 82- It is immediate to show that lA is an associative operation on 
the set of consistent states, with neutral element 0, with upper bound 5'iU5'2, and returning 
a non-empty state whenever 5i U 52 is non-empty. 

Lemma 3.2.2 . Assume i G N and 81, . . . ,8i G S. 

(1) 8iU...U8i C 5iU...U5i 

(2) 8iU ..M8., = (D implies 5i = . . . = 5^ = 0. 

In fact, the whole realizability Semantics is a Monad [12]. In [12], it is proved that our 
realizability Semantics is parametric with respect to the definition we choose for U. Any 
associative operation U, with neutral element and satisfying the two properties of Lemma 
3.2.2, defines a different but sound realizability Semantics, corresponding to a different 
"learning strategy". An immediate consequence of Lemma 3.2.2 is: 

Lemma 3.2.3 . A ssume 8, 81, 82 G S. 

(1) If 8 is consistent with 81,82, then 8 is consistent with 8ilA82- 

(2) If 8 is disjoint with 81,82, then 8 is disjoint with 8iU82- 

For each state of knowledge 8 we assume having a unique constant s denoting it. We 
denote the state denoted by a constant s with |s| and as usual with l.]"-"^ the inverse of |_|; 
that is, ||s||~^ = s. We assume is the state constant denoting the empty state 0; that is, 
|0| = 0. We define with 

rs = T + S + {s I |s| gS} 

the extension of T with one atomic type S denoting S, and a constant s for each state 
8 £ §, and no new reduction rule. We denote states by 8,8',... and state constants by 
s, s', . . .. Any closed normal form of type N,Bool, S in 7s is, respectively, some numeral n, 
some boolean True, False, some state constant s. Computation on states will be defined 
by some suitable set of algebraic reduction rules we call "functional" . 
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Definition 3.2.4 (Functional set of rules). Let C be any set of constants, each one of 
some type Ai —)■... —)■ An — )• for some ^i, . . . , A € {Bool, N, S}. We say that 7^ is a 
functional set of reduction rules for C liTZ consists, for all c G C and all ai : Ai, . . . , a„ : An 
closed normal terms of 7s, of exactly one rule coi . . .an i— )■ a, for some closed normal term 
a: AoiTs- 

Theorem 3.2.5 . Assume that TZ is a functional set of reduction rules for C (def. 3.2.4)- 
Then Ts + C + TZ enjoys: i) strong normalization; ii) weak- Church- Rosser (uniqueness of 
normal forms) for all closed terms of atomic types. 

Proof. (Sketch) For strong normalization, see Berger [13] (the constants s : S and 
c € C are trivially strongly computable). For weak Church- Rosser property, we start from 
the fact that there is the canonical set-theoretical model M of Ts-\-C-\-TZ. The interpretation 
of Bool, N, S in consists of all closed normal form of these types. Arrows and pairs are 
interpreted set-theoretically. Each constant c G C is interpreted by some map /c, defined 
by /c(ai, . . . , a„) = a for all reduction rules (cai . . . a„ a) G TZ. Assume u,v : A are 
closed normal term, A = Bool, N, or S is an atomic type, and u, v are equal in Ts + C -\- TZ, 
in order to prove that u, v are the same term, u, v are equal in A4 because 7W is a model of 
Ts + C -\- TZ. By induction on w we prove that if u; is a closed normal form of atomic type 
T + C -\- TZ, then u; is a numeral, or True, False, or a state constant, and therefore w is 
interpreted by itself in Ai. From u,v equal in A4 we conclude that u,v are the same term 
of Ts + C + 7^. 

□ 

We define two extensions of 7s: an extension 7ciass with symbols denoting the non- 
computable maps Xp,0p and no computable reduction rules, another extension Thcam, 
with the computable approximations XPt4'p of Xp,<t'p, and a computable set of reduction 
rules. We use the elements of 7ciass to represent non-computable realizers, and the elements 
of 7Loarn to represent a computable "approximation" of a realizer. In the next definition, 
we denote terms of type S by />,/)',.• •• 

Definition 3.2.6 . Assume P : n'^+^ Bool is a /c -|- 1-ary predicate of T. We introduce 
the following constants: 

(1) Xp : n'' ^ Bool and Op : n'' ^ N. 

(2) XP : S ^ n'^' ^ Bool and V3p : S ^ n'' ^ N. 

(3) y : S ^ S ^ S. 

(4) Addp : n'^+i ^ S and addp : S ^ n'^+i ^ S. 
We denote iiiJ/0iP2 with pi^ p2- 

(1) Es is the set of all constants xp, ^Pi ^, addp. 

(2) E is the set of all constants Xp, Op, lUJ, Addp. 
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(3) Tciass — 7s + 

(4) A term t € Tciass has state if it has no state constant different from 0. 



Let t = ti . . .tk- We interpret xpst and ippst respectively as a "guess" for the values 
of the oracle and the Skolem map Xp and <t>p for 3y.Pty, guess computed w.r.t. the 
knowledge state denoted by the constant s. There is no set of computable reduction rules 
for the constants 0p,Xp e H, and therefore no set of computable reduction rules for Tciass- 

If si, S2 are state constants, we interpret siiyJs2 as denoting the consistent union |si|Z//|s2|. 
Addp denotes the map constantly equal to the empty state 0. addpsnm denotes the empty 
state if we cannot add the atom {P,n,m) to \s\, either because {P,n,l) G |s| for some 
numeral /, or because Pnm = False; addpsfim denotes the state {{P, fi, m)} otherwise. We 
define a system TLcam with reduction rules over Hs by a functional reduction set T^s. 



Definition 3.2.7 (The System TLearn)- Let s,si,S2 be state constants. Let {P,n,m) be 
an atom. TZg is the following functional set of reduction rules for Hs: 



True if Bin. {P,n,m) G |s| 
False otherwise 



(fpsn I-)- 



addpsfim i— )■ 



m if 3m. {P,n,m) € |s| 
otherwise 

if 3/. (P, n, /) e |s| V Pnm = False 

n, otherwise 



si lyj S2 ss, where S3 is the state constant such that |s3| = |si|^/|s2| 
We define Tleam = Ts + Hs + T^s- 



Remark. TLcam is nothing but Ts with some "syntactic sugar". By Theorem 3.2.5, 
TLcarn is strougly normalizing and has the weak Church-Rosser property for closed term of 
atomic types. TLcam satisfies a Normal Form Property. 



Lemma 3.2.8 (Normal Form Property for TIearn)- Assume A is either an atomic 
type or a product type. Then any closed normal term t G Tlearn of type A is: a numeral 
n : N, or a boolean True, False : Bool, or a state constant s : S, or a pair {u,v) : B x C. 



Proof. (Sketch) By induction over t. For some v, either t is {Xx.u){v), or t is (n, w){v), 
or t is x{v) for some variable x, or t is c{v) for some constant c, and either c = 0, S, True, False, s, 
Rt, if TjTTj is some constant of Ts, or c G Hs. If t = {Xx.u){v), then t has an arrow type 
if i; = 0, while t is not normal if u 7^ 0. If t = {u,w){v), then v = % and we are done. If 
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t = x{v) then t is not closed. The only case left is t = c{u) : A. A is not an arrow type, 
therefore all arguments of c are inu. 1ft = we are done, if t = S{u) we apply the induction 
hypothesis, if i = True, False : Bool or t = s : S or t = {u,v) we are done. Otherwise 
either t = Rxin, f,a)t,ifT{b,ai,a2)t,TTi{v)t, or t = xpW-,'^) : N, or t = ifp{u,w) : N, or 
t = iyj(ui,U2) : S, or t = addp{u,w) : S. The proper subterms n,wi^ . . . : N, 6 : Bool, 
V : Ax B, u,ui,U2 : S of t have atomic or product type and are closed normal. By induction 
hypothesis they are, respectively, a numeral, a boolean, a pair, a state constant. In all cases, 
t is not normal. 

□ 

Let tit2 G TLoarn bc two closcd temis of type S. We abbreviate "the normal forms 
of ti,t2 denote two states which are consistent and disjoint" by: ti,t2 are consistent and 
disjoint. 0, s are consistent and disjoint for every state constant s. The maps denoted by 
lliJ,addp preserve the relation "to be consistent and disjoint". 

Lemma 3.2.9 . Assume s,si,S2 are state constants and {P,n,m) is an atom. 

(1) s, (addpsnm) are consistent and disjoint. 

(2) Assume s, si are consistent and disjoint, and s, S2 are consistent and disjoint. Then 
s, si lyj S2 are consistent and disjoint. 

Proof. (1) If addpsnm denotes the empty state the thesis is immediate. Oth- 
erwise addpsnm denotes {(P, n, m)} and {P,fi,l) ^ \s\ for all numerals /. Then 
{(P, n, m)} is consistent and disjoint with |s|. 
(2) By Lemma 3.2.3. 

□ 

Each (in general, non-computable) term t G Tciass is associated to a set {t[s] \s is a 
state constant} C Tloarn of computable terms we call its "approximations", one for each 
state constant s. 

Definition 3.2.10 (Approximation at state s). Assume t G Tciass and s is a state 
constant. We call "approximation of t at state s" the term t[s] of TLoarn obtained from t 
by replacing each constant Xp with xps, each constant 0p with (fps, each constant Addp 
with addps. 

We interpret any t[s] £ TLcarn as a learning process evaluated w.r.t. the information 
taken from a state constant s (the same s for the whole term). 

Assume t G Tciass is closed, t : S and s is a state constant. Then t[s] is a closed term of 
TLoarni and Its uormal form, by the Normal Form Property 3.2.8, is some state constant s'. 
We conclude t[s] = s' in TLeam- We prove that s, s' are consistent and disjoint. 

Lemma 3.2.11 . Assume s is a state constant, t G Tciass, t : S is closed, and all state 
constants in t are consistent and disjoint with s. 

(1) Ift[s] reduces to t'[s], then all state constants in t' are consistent and disjoint with s. 
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(2) s,t[s] are consistent and disjoint. 

(3) If u £ Tciass, u : S and all state constants in u are 0, then s,u[s] are consistent 
and disjoint. 

Proof. (1) It is enough to consider a one-step reduction. Suppose that t[s] re- 
duces to t'[s] by contraction of a redex r of t[s]. If r is {Xxu)t or RtuvS{w) or 
ifT(6, 01,02) or '7Ti{vi,V2) or XP^^-, or ifpsn, then its contractum r' does not con- 
tain any new state constant; hence, all state constants in t' are consistent and 
disjoint with s. If r is si y S2 or addpsnm, then both s, si and s, S2 are consistent 
and disjoint state constants by hypothesis on t; therefore, by Lemma 3.2.9, in both 
cases s and the contraction of r are consistent and disjoint; so all state constants 
in t' are consistent and disjoint with s. 

(2) Every reduct of t[s] is t'[s] for some t' G Tciass- If t[s] reduces to a normal form 
t'[s] = s', then the only possibility is t' = s'. By the previous point 1, we conclude 
that s' is consistent and disjoint with s. 

(3) By the previous point 2, and the fact that the only state constant in u is 
consistent and disjoint with any s. 

□ 

We introduce now a notion of convergence for families of terms ^ TLoam, 

defined by some t £ Tciass and indexed over a set of state constants {si}jgN. Informally, 
"t convergent" means that the normal form of t[s] eventually stops changing when the 
knowledge state s increases. If si,S2 are state constants, we write si < S2 for C |s2|. 
We say that a sequence {sjjjgN of state constants is a weakly increasing chain of states (is 
w.i. for short), if Si < Sj+i for all i G N. 

Definition 3.2.12 (Convergence). Assume that {sijigN is a w.i. sequence of state 
constants, and u G Tciass- 

(1) u converges in {sJjgN if 3i e N.Vj > i.u[sj] = u[si] in TLoam- 

(2) u converges if u converges in every w.i. sequence of state constants. 

We remark that if u is convergent, we do not ask that u is convergent to the same 
value on all w.i. chain of states. The value learned by u may depend on the information 
contained in the particular chain of state constants by which u gets the knowledge. The 
chain of states, in turn, is selected by the particular definition we use for the "learning 
strategy" U. Different "learning strategies" may learn different values. 

Theorem 3.2.13 (Stability Theorem). Assume t G Tciass is a closed term of atomic 
type A (A £ {Bool,N,S}j. Then t is convergent. 

Proof. (Classical). Assume S is any consistent and possibly infinite set of atoms. We 
define some (in general, not computable) functional reduction set 'TZ{S) for the set H of 
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constants and for Tciass- The reductions for Xp,0p,Addp are those for xp, (/)p, addp in 

arn • 

(1) If (P,n,m) G 5, then (Xpn i-> True), (0pn iH> m) G TliS), else 
(Xpn 1-^ False), (0p?i 0) G 7^(5). 

(2) Addpnm i— > G 7^(5*) if either {P,n,l) G 5 for some numeral I or Pnm = False; 
otherwise, Addpnm i-> n, m)}|~"^ G Tl{S). 

and the reduction for lUJ in 7^(«S') is the reduction for lyj in Tlearn- By theorem 3.2.5, 
Tciass + is strongly normalizing and weak-CR for all closed terms of atomic type, 

for any consistent set of atoms S. For the rest of the proof, let {sjjjgN be a w.i. chain of 
state constants. Assume t G Tciass is a closed term of atomic type A. 

Claim. For any state constant s, the map u i— >■ u[s] is a bijection from the reduction tree of 
t in Tciass + '^(1*1) to the reduction tree of t[s] in Tloarn- 

Proof of the Claim. By induction over the reduction tree of t[s] . Every reduction f3, vr, if Rt, ^ 
over t[s] may be obtained from the same reduction over t. All occurrences of xp^V^p^^^^p 
in the reduction tree of t[s] are of the form xps, ^ps, addps, therefore every reduction over 
Xp, ^Pp, addp may be obtained from the corresponding reduction over Xp, 0p, Addp. 

Assume now a is the (unique, by weak-CR) normal form of t in Tciass + ^(|s|). By the 
Claim, a[s] is the normal form of t[s] in Tloarn- Since a is normal in Tciass + ^(l^l), there 
is no Xp,0p,Addp in a. Thus a and a[s] are the same term: t and t[s] have the same 
normal form respectively in Tciass + '^(kl) and in TLoam- Let {sijigN be a given sequence 
of state constants. Define S^j = UjgN|si|. By strong normalization, the reduction tree of 
t in Tciass + T^iSu)) is finite. Therefore in this reduction tree are used only finitely many 
reduction rules from TZ{Suj), and for some numeral n it is equal to the reduction tree of t 
in Tciass + T^{\sn\), and in Tciass + T^{\sm\) for all m > n. We deduce that for all m > n the 
normal forms of t in Tciass + '^(I'Sml) are the same. Thus, the normal form in TLoam of all 
t[sm] with m > n are the same, as we wished to show. □ 

Remark 3.2.14 . The idea of the proof of theorem 3.2.13 corresponds exactly to the 
intuition of the introduction. During any computation, the oracles Xp and 0p are consulted 
a finite number of times and hence asked for a finite number of values. When our state of 
knowledge is great enough, we can substitute the oracles with their approximation xps and 
(fps for some state constant s, and we will obtain the same oracle values and hence the 
same results. 

The proof, though non constructive, is short and explains well why the result is true. 
However, provided we replace the notion of convergence used in this chapter with the 
intuitionistic notion introduced in [7], we are able to reformulate and prove theorem 3.2.13 in 
a purely intuitionistic way, achieving thus a constructive description of learning in HA+EMi. 
Being the intuitionistic proof much more elaborated and less intuitive than the present one 
and connected with other foundationally interesting results, it will be the subject of chapter 
5. 

Our proof of convergence follows the pattern of Avigad's one in [5]. A closed term 
t G Tciass of atomic type and in the constants ci, . . . , Cn G H, may be seen as a functional Ft 
which maps functions /i, ...,/„ of the same type of ci, . . . c„ into an object of atomic type: 
. . . , fn) is defined as the normal form of t in Tciass + T^, where TZ = {cjOi . . . i->- 
a I /i(ai, . . . , an) = a and i G {1, . . . , n}}. Ft is continuous in the sense of Avigad. Moreover, 
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since Xp and Addp have a set-theoretical definition in terms of (t>p, we may assume Ft 
depends only on the functions which define in TZ the reduction rules for cj>p^ , . . . 0p^. Then, 
if t is of type S, it is not difficult to see that Ft represents an update procedure with respect 
to any of its argument. The fact that Ft is an update procedure implies convergence for t 
and the zero theorem 3.2.15. 

As last result of this section, we prove that if we start from any state constant s and we 
repeatedly apply any closed term t : S of Tciass of state (see definition 3.2.6), we obtain 
a "zero" of t, that is a state constant Sn such that t[sn] = 0- We interpret this by saying 
that any term t represents a terminating learning process. 

Theorem 3.2.15 (Zero Theorem). Let t : S be a closed term of Tciass of state and s 
any state constant. Define, by induction on n, a sequence {snjnGN of state constants such 
that: sq = s and Sn+i = Sn ^ t[sn]- Then, there exists an n such that t[sn] = 0. 

Proof, sq, si,S2, ■ ■ ■ is a weakly increasing chain of state constants by construction. By 
theorem 3.2.13, t converges over this chain: there exists A; G N such that for every j > k, 
t[sj] = t[sk]- By choice of k 

Sk+2 = Sk+l ^ t[sk+i] 

= {Sk^t[sk])Vi}t[sk+l] 

= (sfciyjtN)iyjtN 
= sfc y t[sk] 

Since Sk+2 = ■Sfc+i ^tnd s^+i, t[sk+i] are consistent and disjoint by lemma 3.2.11, we conclude 

t[Sk+l] = 0. 

□ 

3.3. An Interactive Learning-Based Notion of Realizability 

In this section we introduce the notion of realizability for HA-|- EMi, Heyting Arithmetic 
plus Excluded Middle on S^-formulas, then we prove our main Theorem, the Adequacy 
Theorem: "if a closed arithmetical formula is provable in HA + EMi, then it is realizable". 

We first define the formal system HA-|- EMi, from now on "Extended EMi Arithmetic". 
We represent atomic predicates of HA + EMi with (in general, non-computable) closed 
terms of Tciass of type Bool. Terms of HA + EMi may include function symbols Xp, 0p 
denoting non-computable functions: oracles and Skolem maps for E^-formulas Bx.Pxfi, 
with P predicate of T. We remark that our realizability can be formulated already for the 
standard language of Arithmetic: we add non computable functions to the language for 
greater generality. We assume having in T some terms ^booi: Bool, Bool — )• Bool, -ibooi : 
Bool — )■ Bool, . . ., implementing boolean connectives. If ti, . . . , t„, i € T have type Bool and 
are made from free variables all of type Bool, using boolean connectives, we say that t is a 
tautological consequence of ti, . . . , in T (a tautology if n = 0) if all boolean assignments 
making ti, . . . ,tn equal to True in T also make t equal to True in T. 
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Definition 3.3.1 (Extended EMi Intuitionistic Arithmetic: HA + EMi). The lan- 
guage £ciass of HA + EMi is defined as follows. 

(1) The terms of >Cciass are all t £ Tciass with state 0, such that t : N and FV{t) C 
{x^, . . . , x^} for some xi, . . . , x„. 

(2) The atomic formulas of £ciass are all Qti . . .tn G Tciass i for some Q : N" — )• Bool 
dosed term of Tciass of state 0, and some terms ti, . . . , t„ of >Cciass- 

(3) The formulas of /^ciass are built from atomic formulas of £ciass by the connectives 
V, A, — >■ V, 3 as usual. 

A formula of HA is a formula of HA + EMi in which all predicates and terms are terms 
of T. 

Deduction rules for HA + EMi are as in van Dalen [15], with: (i) an axiom schema for 
EMi; (ii) the induction rule; (in ) as Post rules: all axioms of equality and ordering on N, 
all equational axioms of T, and one schema for each tautological consequences of T. (iv) the 
axiom schemas for oracles: P{t,t) =>booi Xpt* and for Skolem maps: Xpt =^>booi P{ti {'^P't))^ 
for any predicate P of T. 

We denote with _L the atomic formula False and will sometimes write a generic atomic 
formula as . . . , tn) rather than in the form Pti . . .tn- Finally, since any arithmetical 
formula has only variables of type N, we shall freely omit their types, writing for instance 
yx.Ain place of\/x^.A. Post rules cover many rules with atomic assumptions and conclusion 
as we find useful, for example, the rule: "if f{z) < then f{z) = 0". 

We defined =^booi: Bool, Bool — > Bool as a term implementing implication, therefore, 
to be accurate, the axiom P{ti, . . . ,t„,t) =^>booi Xpti . . .tn is not an implication between 
two atomic formulas, but it is equal to the single atomic formula Qti . . . tnt, where 

(5 = Ax" . . . Ax"+i. =^Bool {Pxi . . . XnX,i+i)(XpXi . . . Xn+l) 

Similarly, ^BooiP{t, t) will denote a single atomic formula. Any atomic formula A of 
^ciass is a boolean term of Tciass i therefore for any state constant s we may form the "finite 
approximation" A[s] : Bool, A[s] € TLcarn of A. In A[s] we replace all oracles Xp and all 
Skolem maps 0p we have in A by their finite approximation xpSt4'ps, computed with 
respect to the state constant s. We denote with /^Lcam the set of all expressions A[s] with 
A € >Cciass and s a state constant. All A[s] € /^Loam may be interpreted by first order 
arithmetical formulas having all closed atomic subformulas decidable. 

Using the metaphor explained in the introduction, we use a set of falsifiable hypotheses 
determined by s to predict a computable truth value A[s] : Bool in TLcam for an atomic for- 
mula A G >Cciass that we cannot effectively evaluate. Our definition of realizability provides 
a formal semantics for the Extended Intuitionistic Arithmetic HA-|- EMi, and therefore also 
for the more usual language of Arithmetic HA, in which all functions represent recursive 
maps. 

Definition 3.3.2 (Types for realizers). For each arithmetical formula A we define a 
type 1^1 of T by induction on A: 
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(1) 


\P{h,.. 


■ ) ^n) 1 — S, 


(2) 


\AAB\ 


= 1^1 X l^l- 


(3) 


\A\J B\ 


= Bool X (1^1 


(4) 


\A^ B 


= 1^1 ^ 1^1, 


(5) 


\yxA\ = 


N \A\, 


(6) 


\3xA\ = 


N X 1^1 



We now define the realizability relation t llh A, where t € Tciass) A £ £ciass) t has state 
and t : \A\. 

Definition 3.3.3 (Realizability). Assume s is a state constant, t e Tciass is a closed 
term of state 0, C € -Cciass is a closed formula, and t : \C\. Let t = ti, . . . ,tn ■ ^- We define 
first the relation t llhg C by induction and by cases according to the form of C: 

(1) t llhs P{t) if and only if t[s] = in TLcam implies P(t)[s] = True 

(2) t \\\-s AAB if and only if not llh^ A and nit llh^ B 

(3) t llhs ^4 V -B if and only if either pot[s] = True in TLcam and pit llh^ A, or 
Pot[s] = False in TLcam and p2t IW-g B 

(4) t \\\-s ^ ^ if and only if for ah u, if u llh^ A, then tu llh^ B 

(5) t \\\-s VxA if and only if for all numerals n, tn llh^ ^[n/a::] 

(6) t llhs 3xA if and only for some numeral n, nQt[s] = n in TLcam and vrit llh^ y4[n/x] 

We define t llh A if and only if for all state constants s, t llh^ A. 

The definition of llh formalizes all the idea we sketched in the introduction. A realizer 
is a term t of Tciass, possibly containing the non-computable functions Xp,<t)p; if such 
functions were computable, t would be an intuitionistic realizer. Since in general t is not 
computable, we calculate its approximation t[s\ at state s, which is a term of TLcam, and we 
require it to satisfy the indexed- by-state realizability clauses. Realizers of disjunctions and 
existential statements provide a witness, which is an individual depending on an actual state 
of knowledge, representing all the hypotheses used to approximate the non-computable. The 
actual behavior of a realizer depends upon the current state of knowledge. The state is used 
only when there is relevant information about the truth of a given formula to be computed: 
the truth value -P(ti, . . . , tn)[s\ of an atomic formula and the disjunctive witness pot[s\ and 
the existential witness 7roti[s] are computed w.r.t. the constant state s. A realizer t of 
Ay B uses s to predict which one between A and B is realizable (if pot[s\ = True then 
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A is realizable, and if po^l-s] = False then B is realizable). A realizer u of 3xA uses s to 
predict that 7roii[s] equals an n, some witness for 3xA (i.e. that A[n/x] is realizable). These 
predictions need not be always correct; hence, it is possible that a realized atomic formula 
is actually false; we may have t llh^ P and P[s] = False in TLoam- If an atomic formula, 
although predicted to be true, is indeed false, then we have encountered a counterexample 
and so our theory is wrong, our approximation still inadequate; in this case, t[s] ^ by 
definition of t llhg P, and the atomic realizer t takes s and extends it to a larger state 
■s', union of s and t[s]. That is to say: if something goes wrong, we must learn from our 
mistakes. The point is that after every learning, the actual state of knowledge grows, and 
if we ask to the same realizer new predictions, we will obtain "better" answers. 

Indeed, we can say more about this last point. Suppose for instance that t \\\- AV B 
and let {sjjjgN be a w.i. sequence. Then, since t : Bool x |^| x \B\, then pot : Bool is a 
closed term of Tciassi converging in {sjjjgN to a boolean; thus the sequence of predictions 
{po^[si]}«eN eventually stabilizes, and hence a witness is eventually learned in the limit. 

In the atomic case, in order to have t IW-g P{ti, . . . , tn), we require that if t[s] = 0, then 
P{ti, . . . ,tn)[s] = True in TLcarn- That is to say: if t has no new information to add to s, 
then t must assure the truth of P{ti, . . . , tn) w.r.t. s. By the zero theorem 3.2.15, when t : S 
is closed, there is plenty of state constants s such that t[s] = 0; hence search for truth will 
be for us computation of a zero, driven by the excluded-middle instances and the Skolem 
axioms used by the proof, rather than exhaustive search for counterexamples. In chapter 5 
we will prove that, actually, zeros for terms of Tciass can be computed by learning processes 
whose length can be bounded through constructive reasoning. 

It is useful to give a slightly different definition of indexed realizability, which in some 
situations is slightly easier to reason with. The difference with definition 4.2.4 is only that 
the relation we are going to define now is between terms of TLeam and formulas of /^Leam, 
which are from the beginning approximations at some state s respectively of terms of Tciass 
and formulas of -Cciass- 

Definition 3.3.4 (Variant of Indexed Realizability). Let s be a state constant. 
Assume t € Tloarn and A G >CLcarn are of the form t = t'[s], A = A' [s] for some closed 
i' £ Tciass of state and some closed A' £ £ciass- We define t Ih^ A for any state constant 
s by induction on A. 

(1) t W-g P{ti, . . . ,tn) if and only if t = in TLcarn implies P{ti, . . . ,tn) = True 

(2) t\\-s A A B ii and only if vrot Ih^ A and nit Ih^ B 

(3) tW-gAVBif and only if: either pot = True in TLoam and pit Ih^ A, or pQt = False 
and p2t W-g B 

(4) t W-g A ^ B if and only if for all u, if n Ih^ A, then tuW-g B 

(5) t W-g yxA if and only if for all numerals n, tn Ih^ 74[n/a;] 

(6) t \\-g 3xA if and only if for some numeral n not = n in TLoam and vrit Ih^ A[n/x] 
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The realizability relation is compatible with equality in TLoam^ 
Lemma 3.3.5 . If ti Ihg A[ni/x], ti = t2 and ui = U2 in Tieam, then t2 IH^ ^[u2/x] 
Proof. By straightforward induction on A. 

□ 

We can now characterize llh in the following way. 

Lemma 3.3.6 (Alternative Characterization of Realizability). Assumet € Tciass 
is a closed term, A E Cciass is a closed formula, and t : \A\. Then 

t \\\- A if and only if for all state constants s, t[s]\\-s A[s] 

Proof. By definition unfolding and by induction on A, one shows that t llhs A if and 
only if t[s] Ih^ A[s]. 

□ 

Example 3.3.7 . The most remarkable feature of our Realizability Semantics is the exis- 
tence of a realizer Ep for EMi. Assume that P is a predicate of T and define 

Ep := Xa^{Xpa, {<t>pd, 0), An" Addpan) 

Indeed Ep realizes its associated instance of EMi. 

Proposition 3.3.8 (Realizer Ep of EMi). 

Ep llh Vx. 3y P(x, y) V Vy-.Booi-P(^, y) 

Proof. Let m be a vector of numerals. Ep?7i[s] is equal to 

{xpsm, {ifpsrh, 0), An"* addpsmn) 

and we want to prove that 

Epm[s] 3y P{rh,y) V yy^BooiP{m,y) 

We have poEprn[s] = xpsm in Tloarn- There are two cases. 

(1) xpsrh = True. Then (P, m, n) G \s\ for some numeral n such that P(rh, n) = True, 
and we have to prove 

piEpm[s] hs P{rn,y) 

By definition of ippsfh 

piEpm[s] = {ippsm,0) = (n,0) 

Thus 

7ro(piEpm)[s] = 7ro(n, 0) = n 

and 

7ri(piEpm)[s] Ihs P{rh,n) 
because P{rh,n) = True. We conclude 

PiEp7n[s] 3y P{m,y) 
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(2) xp^'itT' = False. Then {P^rh^l) ^ \s\ for all numerals I. We have to prove 

P2^pm[s\ = Xn sddpsfhn Ih^ Vy-iBooi-Pl^^i, y) 

i.e. that given any numeral n 

addpsmn Ih^ -^BooiPimn-tn) 

By the definition of realizer in this case, we have to assume that addpsmn = 0, 
and prove that -'booi-P("^, = True. The substitution {.)[s\ has an empty effect 
over P{rh,n), therefore we have to prove that ^BooiPi'fTii'n) = True, that is, that 
P{rh,n) = False. Assume for contradiction that P[m,n) = True. We already 
proved that {P,rn,l) |s|, for all numerals from this and P{rh,n) = True we 
deduce that by definition addpsmn = \{{P,'in,n)}\~^ , contradiction. 

□ 

Ep works according to the ideas we sketched in the introduction. It uses xp to make 
predictions about which one between 3y P{m, y) and V?/-ibooi-P("^, v) is true, xp-, iii turn, 
relies on the constant s denoting the actual state to make its own prediction. If xpsm = 
False, given any n, -^BooiP{'m, n) is predicted to be true; if it is not the case, we have 
a counterexample and addp requires to extend the state with (P, m,n). On the contrary, 
if xpsm = True, there is unquestionable evidence that 3yP{m,y) holds; namely, there is 
some numeral n such that {P,m,n) is in s; then ipp is called, and it returns ippsrh = n. 

This is the basic mechanism by which we implement learning: every state extension is 
linked with an assumption about an instance of EMi which we used and turned out to be 
wrong (this is the only way to come across a counterexample); in next computations, the 
actual state will be bigger, the realizer will not do the same error, and hence will be "wiser" . 

As usual for a realizability interpretation, we may extract from any realizer t llh \lx3y.P{x, 
with P E T, some recursive map / from the set of numerals to the set of numerals, such 
that P{n, f{n)) for all numerals n. 

Example 3.3.9 (Program Extraction via Learning Based Realizability). Let t 
be a term of Tciass and suppose that t llh Vx^By^Pxy, with P atomic. Then, from t one can 
effectively define a recursive function / from the set of numerals to the set of numerals such 
that for every numeral n, Pn(/(n)) = True. 

Proof. Let 

V := Am" TTiitm) 

V is of type N ^ S. By zero theorem 3.2.15, there exists a recursive function zero from the 
set of numerals to the set of state constants such that t'n[zero(n)] = for every numeral n. 
Define / as the function 

m i-T- 7ro(tm)[zero(m)] 

and fix a numeral n. By unfolding the definition of realizability with respect to the state 
zero(n), we have that 

tn l!^zero{n) ^V^Pny 

and hence 

7ri(tn) llh^ero(n) Pn{f{n)) 
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that is to say 

vn[zero{n)] = =^ Pn{f{n)) = True 

and therefore 

Pn{f{n)) = True 

which is the thesis. 

□ 

Remark 3.3.10 . In chapter 5 we shall prove that the map / constructed in example 3.3.9 is 
even definable in Godel's T. This result formally proves that our realizability interpretation 
is a constructive semantics and that / is not a brute force search algorithm. More precisely, 
we can argue as follows. The numeral f{n) is computed by finding a zero of vn = 7ri(tn), 
i.e. a state s such that 7ri(tn)[s] = 0. By the Zero theorem 3.2.15, this zero is computed 
step by step by constructing the sequence sq = 0, s„+i = s„ lU) t[sn] and stopping at the 
first m such that 'Ki{tn)[sm] = 0- 

First, we observe that each portion of Sm is efficiently constructed: for each n, the state 
Sn is efficiently extended to Sn+i, through the addition of new oracle values learned by 
t by counterexamples, i.e. by the falsification of some excluded middle or Skolem axiom 
instances. No brute force search whatsoever, thus, for new oracle values: they are all 
efficiently produced by t[s„], which, modulo some trivial coding, is a term of Godel's T and 
just cannot search blindly for oracle values, since it is a primitive recursive functional of 
finite type. 

Secondly, an upper bound to m can be computed in Godel's T, as proven in chapter 5, 
theorem 5.6.2. Moreover, this upper bound results from a constructive proof of the Zero 
theorem. Hence, Sm and thus f{n) can be defined by a primitive recursive functional of 
finite type, which, again, by construction cannot explore blindly the infinite search space of 
the knowledge states in order to find a zero. 

Moreover, from the low level computational point of view and in the language of e- 
substitution method, our realizers represent convergent procedures to find out a "solving 
substitution", i.e. a state representing an approximation of Skolem functions (i.e., e-terms) 
which makes true the Skolem axioms instances used in a proof of an existential statement. 
The advantage of our semantics is the possibility of defining such procedures directly from 
high level proofs, by means of Curry-Howard correspondence, hence avoiding the round- 
about route which forces to use a quantifier free deduction system. In the case of a provable 
formula in the language of Peano Arithmetic (that is, one not containing the symbols Xp 
or 0p) we do not need at all to modify the language of its proof and to use the Skolem 
axioms x^V- 

Now we explain how to turn each proof I? of a formula A € £ciass in HA + EMi into 
a realizers T>* of the same A. By induction on D, we define a "decoration with realizers" 
pReai Qf which each formula i? of "D is replaced by a new statement u\- B, for some 

u € Tciass of state 0. If t h ^ is the conclusion of T>^°^^, we set "D* = t. Then we will prove 
that if V is closed and without assumptions, then D* € Tciass and P* llh A. The decoration 
pReai q£ p with realizers is completely standard: we have new realizers only for EMi and 
for atomic formulas. For notation simplicity, if Xj is the label for the set of occurrences of 
some assumption Ai of V, we use Xi also as a name of one free variable in V* of type \Ai\. 
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If T is any type of 7s, we denote with d-^ a dummy term of type T, defined by = 0, 
rfBooi ^ Yslse, = 0, d^^^ = A_^.d^ (with _^ any variable of type A), d^""^ = {d^,d^). 

Definition 3.3.11 (Term Assignment Rules for HA + EMi). Assume P is a proof 
of yl G -Cciass in HA + EMi, with free assumptions Ai, . . . , A„ denoted by proof variables 
x^^ , • • • , x^" and free integer variables a^, . . . , a^. By induction on V, we define a deco- 
rated proof-tree pi^"^', in which each formula B is replaced hy u\- B for some u E Tciassi ^-nd 
the conclusion A with some t\- A, with FV{t) C {x^"^^', . . . , x^"^^', a^, . . . , oi^}. Eventually 
we set V* = t. 



(1) h A if V consists of a single free assumption A S >Cciass labeled x^. 

u\-A t\-B u\-AaB u\-AaB 

^ ' {U,t)h AAB TTqU h A TTiU h B 

uh A^ B th A n h .B 

^ ' ^ITB \x\^\u^ A^B 



(4) 



(5) 



u\- A u\- B 

{True,u,d^) \- A\J B {False, d^,u) \- Ay B 

\f Pqu then {Xx^^^wi)(j>iu) else {Xx^^^W2){p2u) H C 

where and d^ are dummy closed terms of Tciass of type |^| and \B\ 

u h VaA u\- A 



(6) 



ut h A[t/a] Aa™n h ^aA 

where t is a term of >Cciass and a" does not occur free in any free assumption 
B of the subproof of V of conclusion A. 

u h A[t/a''] nh3a".yl thC 

(t,u) h 3a™. ^ (Aa^'Axl^l t)(^o^i)(vrin) h C 



(7) 
(8) 



where a" is not free in C nor in any free assumption B different from A in the 
subproof of T> of conclusion C. 

u h A{0) V h Va.A(a) ^ ^(S(a)) 



Xa^Ruva h VaA 

fil h yli ^2 H ^2 • • • Un\- Ar. 



til lyj n2 lyj • • • lyj H A 

where n > and Ai, A2, ■ ■ ■ , An, A are atomic formulas of >Cciass5 and the rule 
is a Post rule for equality or ordering, or a tautological consequence. 



(9) h A 

where A is an atomic axiom of HA + EMi (an axiom of equality or of ordering 
or a tautology or an equation of T) 
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(10) Ep h Vf . 3y P{x, y) V Vy-Booi/'(x, y) 

where P is a predicate of T. 

(11) Addpi"; t h P{t, t) ^Booi Xpf , X- Axiom 

(12) h Xpf^Booi P(i', («t)pt)) , V?- Axiom 

The term decorating the conclusion of a Post rule is of the form lyj • • • lU) u„ . In this 
case, we have n different realizers, whose learning capabilities are put together through a 
sort of union. By Lemma 3.2.2.2, if lyj • • • lU) ii„ [s] = 0, then ui[s\ = ... = Un[s\ = 0, 
i.e. ah Ui "have nothing to learn". In that case, each m must guarantee Ai to be true, and 
therefore the conclusion of the Post rule is true, because true premises ^i, . . . , A„ speh a 
true conclusion A. 

We now prove our main theorem, that every theorem of HA + EMi is realizable. 

Theorem 3.3.12 (Adequacy Theorem). Suppose that V is a proof of A in the system 
HA + EMi with free assumptions xf^,...,x^" and free variables ai : N,...,ak '■ N. Let 
w = T>* . For all state constants s and for all numerals ni, . . . , n^, if 

ti[s] Ihs Ai[ni/ai ■ ■ ■ nfc/afc][s], . . . ,t„[s] Ih^ A„[ni/ai • • • nfc/afc][s] 

then 

'wfti/x'i^'' • • • tn/x.!,"^"! ni/ai ■ ■■nk/ak][s\ Ih^ A[ni/ai ■ ■ ■ nk/ak][s] 

Proof. Notation: for any term v and formula B, we denote 

v[ti/x^^'^^ ■ ■■tn/x\^''\ ni/ai ■ ■ ■ nk/ak][s] 

with V and B[ni/ai ■ ■ ■ nfc/afc][s] with B. We have \B\ = \B\ for all formulas B. We denote 
with = the provable equality in TLcam- We proceed by induction on w. Consider the last 
rule in the derivation D: 

(1) If it is the rule for variables, then w = x^"^'' = x'"^'' and A = Ai. So w = tt Ih^ 
'Ai = A. 

(2) If it is the A/ rule, then w = {u,t) , A = B A C , u h B and t h C. Therefore, 
W = {u,t). By induction hypothesis, vroIfJ = u \\-s B and niw = t Ih^ C; so, by 
definition, w W-g B A C = A. 

(3) If it is a AE rule, say left, then w = ttqu and u\- AaB. Sow = ttqu Ihg A, because 
u\\-s A A B hy induction hypothesis. 

(4) If it is the — )■ E rule, then w = ut, u \- B ^ A and t \- B. So w = ut Ih^ A, for 
u W-g B ^ A and tW-gBhy induction hypothesis. 

(5) If it is the I rule, then w = Xx^^^u, A = B ^ C and n h C. Thus, w = Xx^^^u. 
Suppose now that t Ihg B; by induction hypothesis on u, wt = u[t/x\^\] Ih, C. 



38 3. INTERACTIVE LEARNING-BASED REALIZABILITY FOR HEYTING ARITHMETIC WITH EMi 



(6) If it is a V/ rule, say left, then w = (True, d*^), A = B V C and u \- B. So, 
w = (True, M, (i*^) and hence pqw = True. We indeed verify that piW = u\\-sB 
with the help of induction hypothesis. 

(7) If it is a yE rule, then 

u; = if pqu then {Xx^^^wi)piu else {Xy^'-'^W2)p2U 

andu\- By C,wi\- D,W2\- D,A = D. So, 

w = \f pou then {Xx^^^wi)piu else {Xy^'~'^W2)p2U 

Assume pqu = True. Then by inductive hypothesis piu Ih^ B, and again by in- 
duction hypothesis, W = HJi [piu/x'^l] Ih^ D. Symmetrically, if pqu = False, then 
wl^s D. 

(8) If it is the rule, then w = ut, A = B[t/a] and u h \/aB. So, w = ut. For 
some numeral n we have n = t. By inductive hypothesis u Ih^ \/aB, therefore 
ut = un \hs 'B[n/a] = 'B\i/a] = A 

(9) If it is the V/ rule, then w = Xa^u, A = VaB and u\- B. So, w = Xa^u. Let n be 
a numeral; we have to prove that wn = u[n/a\ Ih^ B[n/a\, which is true, indeed, 
by induction hypothesis. 

(10) If it is the 3E rule, then w = {Xa^ Xx\^h){Trou){TTiu), t h A and u h 3a^.B. Assume 
n = ttqu, for some numeral n. Then 

t[n/a",^iu/xl^["/"'']l] Ih, A[n/a] = A 

by inductive hypothesis, whose application being justified by the fact, also by 
induction, that u Ih^ 3a^.B and hence vriu Ihg B[n/a^]. We thus obtain 

(11) If it is the 3/ rule, then w = {t,u), A = 3aB, u h B[t/a]. So, W = {t,u); and, 
indeed, ttiw = u Ih^ B[K{yw/a\ = B'\^/a\ since by induction hypothesis u Ih^ B\f/a\. 

(12) If it is the induction rule, then w = Xa^ Ruva, A = \/aB, u h B{{)) and 
V h \ta.B{a) B{S{a)). So, w = Aa^RmJa. Now let n be a numeral. A 
plain induction on n shows that wn = Rwvn W-g B[n/Q\, for u Ih^ -B(O) and 
vi Ihs B{i) — )• B{S{i)) for all numerals i by induction hypothesis. 

(13) If it is a Post rule, then w = ui^U2^- ■ -^Un and Ui h Ai. So, w = uimi2^- • -^Un- 
Suppose now that w\s\ = 0; then we have to prove that A = True. It suf- 
fices to prove that Ai = A2 = • • • = An = True. By Lemma 3.2.2 we have 
ui = ■ ■ ■ = Un = and by induction hypothesis Ai = ■ ■ ■ = An = True, since 
Ui Ihs Ai, for i = 1, . . . , n. 

(14) If it is a x-axiom rule, then w = Addpti . . . tnt and 

A = Piti,...,tn,t) ^Xpti...tn 
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Let t — ti, . . . jt^. For some numeral m we have m — t. Suppose by contra- 
diction that w = and P{t,t) = P{t,m) = True and Xpst = False. From 
Xpst = False we get {P, t, m!) ^ s for all numerals m' . We deduce w = add pstm = 
[{(P, m)}|~^, contradiction. 

(15) w realizes an EMi axiom: this is Proposition 3.3.8. 

(16) If it is a 93-axiom rule, then = and 

A = Xpti ...tn^ P{ti, . . . , t„, (cDpti . . . tn)) 

We have w = 0. Let us denote t = ti . . .tn- Suppose that XP^t = True. Then 
for some numeral m we have {P,t,m) € s and Ptm = True and <ppst = m. By 
definition of (pp we have 

P{t, (ifpsi)) = True 
We conclude that A = True. 

□ 

Corollary 3.3.13 . If A is a closed formula provable in HA + EMi, then there exists 
t G Tciass such that t llh A. 

3.4. Conclusion and further works 

Many notions of realizability for Classical Logic already exists. A notion similar to our 
one in spirit and motivations is Goodman's notion of Relative realizability [24]. However, 
there is an intrinsic difference between our solution and Goodman's solution. Goodman uses 
forcing to obtain a "static" description of learning. His "possible worlds" are learning states, 
but there is no explicit operation updating a world to a larger word. The dynamic aspect of 
learning (which is represented by a winning strategy in Game Semantics) is therefore lost. 
Using our realizability model, a realizer of an atomic formula, instead of being a trivial map, 
is a map extending worlds, whose fixed points are the worlds in which the atomic formula 
is true. Extending a world represents, in our realizability Semantics, the idea of "learning 
by trial-and-error" that we have in game semantics, while fixed points represent the final 
state of the game. 

A second notion related to our realizability Semantics is Avigad's idea of "update proce- 
dure" [5]. A state s in our chapter corresponds to a finite model of skolem maps in Avigad. 
An "update procedure" is a construction "steering" the future evolution of a finite partial 
model s of skolem maps, to which our individuals belong, in a wanted direction. The main 
difference with our work is that we express this idea formally, by interpreting an "update 
procedure" as a realizer (in the sense of Kreisel) for a Skolem axiom. Another important 
difference is that our realizability relation is defined for all first-order formulas with Skolem 
maps, while the theory of "update procedures" is defined only for quantifier-free formulas 
with Skolem maps. 

Another difference with the other realizability or Kripke models for Classical Logic is 
in the notion of individual and in the equality between individuals. Assume that m is the 
output of a skolem map for 3y.P{n,y), with P decidable, and m = {m[s]\s € S} a family of 
values depending on the finite partial model s. Then our realizer for Skolem axioms "steers" 



40 3. INTERACTIVE LEARNING-BASED REALIZABILITY FOR HEYTING ARITHMETIC WITH EMi 



the evolution of s towards some universe in which the axiom 3y.P{n,y) =^ P(n, m[s]) is 
true. Modifying the evolution of s may modify the value of m[s]. In our realizability Seman- 
tics we introduce a notion of individuality which is "dynamical" (depending on a state s) 
and "interactive" (the value of the individual depends on what a realizer does). This second 
aspect is new. A realizer may "try" to equate an individual a = {a[s]\s G S} with another 
individual b = {b[s]\s E S}. Whenever this is possible, the realizer defines a construction 
over the evolution of the universe s producing such an effect, while a random evolution of 
s (without an "interaction" with the realizer) does not guarantee that eventually we have 
a[s] = b[s]. This is why, in our realizability model, even equality among concrete objects is 
not a "statical" fact, but it is the effect of applying a realizer (which is a construction over 
the evolution of the state or "world" s). In the other models either equality is "static", or, 
even when it is "dynamical" , and it changes with time, it is not "interactive" : the final truth 
value of an equality is not the effect of the application of the realizer, but it is eventually 
the same in all future evolutions of the current world. 

Many aspects of this chapter will require some further work. A challenging idea is to 
iterate the construction we had for EMi, in order to provide a learning model for the entire 
classical Arithmetic. In this case the leading concepts would be the game-theoretical notion 
of "level of backtracking", introduced in [9] and [11], a notion related to the more informal 
notion of non-monotonic learning. 

Another aspect deserving further work is comparing the programs extracted from clas- 
sical proofs with our method and with other methods, say, with Friedman A-translation. 
Our interpretation, explaining in term of learning how the extracted program work, should 
allow us to modify and improve the extracted program in a way impossible for the more 
formal (but very elegant) ^d-translation. 

We remarked that our interpretation is implicitly parametric with respect to the op- 
eration U merging the realizers of two atomic formulas. As explained in [12], by choos- 
ing different variant of this operation we may study different evaluation strategies for the 
extracted programs: sequential and parallel, left-to-right and right-to- left, confluent and 
non-confluent. We would like to study whether by choosing a particular evaluation strategy 
we may extract a more efficient program. 



CHAPTER 4 



Learning Based Realizability and 1-Backtracking Games 



Abstract. We prove a soundness and completeness result for learning based realizability 
with respect to 1-Backtracking Coquand game semantics. First, we prove that interactive 
learning based classical realizability is sound with respect to Coquand game semantics. In 
particular, any realizer of an implication-and-negation-free arithmetical formula embodies 
a winning recursive strategy for the 1-Backtracking version of Tarski games. We also give 
examples of realizer and winning strategy extraction for some classical proofs. Secondly, 
we extend our notion of realizability to a total recursive learning based realizability and 
show that the notion is complete with respect to Coquand semantics, when it is restricted 
to 1-Backtracking games. 



4.1. Introduction 

In this chapter we show that learning based realizabihty (see chapter 3) relates to 1- 
Backtracking Tarski games as intuitionistic realizability (see Kleene [31]) relates to Tarski 
games, when one considers implication-and-negation-free formulas. The relationship we 
refer to is between realizability on one hand, and existence of winning strategies on the 
other. In particular, it is known that a negation-and-implication-free arithmetical formula 
is Kleene realizable if and only if Eloise has a recursive winning strategy in the associated 
Tarski game. We show as well that an implication-and-negation-free arithmetical formula 
is "learning realizable" if and only if Eloise has recursive winning strategy in the associated 
1-Backtracking Tarski game. 

It is well known that Tarski games (which were actually introduced by Hintikka, see 
[29] and definition 4.3.3) are just a simple way of rephrasing the concept of classical truth 
in terms of a game between two players - the first one, Eloise, trying to show the truth of 
a formula, the second, Abelard, its falsehood - and that a Kleene realizer gives a recursive 
winning strategy to the first player. The result is quite expected: since a realizer gives a 
way of computing all the information about the truth of a formula, the player trying to 
prove the truth of that formula has a recursive winning strategy. However, not at all any 
classically provable arithmetical formula allows a winning recursive strategy for that player; 
otherwise, the decidability of the Halting problem would follow. 

In [14], Coquand introduced a new game semantics for Peano Arithmetic, centered on 
the concept of "Backtracking Tarski game": a special Tarski game in which players have 
the additional possibility of correcting their moves and backtracking to a previous position 
of the game anytime they wish. Coquand then showed that for any provable negation- 
and-implication-free arithmetical formula A, Eloise has a recursive winning strategy in the 
Backtracking Tarski game associated to A. Remarkably, a proof in Peano Arithmetic thus 
hides a non trivial computational content that can be described as a recursive strategy that 
produces witnesses in classical Arithmetic by interaction and learning. 
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In the first part of this chapter, we show that learning based reahzers have direct 
interpretation as recursive winning strategies in 1-Backtracking Tarski games (which are a 
particular case of Coquand games: see Berardi et al [9] and definition 4.3.2 below). The 
result was wished, because interactive learning based realizers, by design, are similar to 
strategies in games with backtracking: they improve their computational ability by learning 
from interaction and counterexamples in a convergent way; eventually, they gather enough 
information about the truth of a formula to win its associated game. 

An interesting but incomplete step towards our result was the Hayashi realizability 
[27]. Indeed, a realizer in the sense of Hayashi represents a recursive winning strategy in 
1-Backtracking games. However, from the computational point of view, Hayashi realizers 
do not relate to 1-Backtracking games in a significant way: Hayashi winning strategies work 
by exhaustive search and, actually, do not learn from the game and from the interaction 
with the other player. As a result of this issue, constructive upper bounds on the length 
of games cannot be obtained, whereas using our realizability it is possible. For example, in 
the case of the 1-Backtracking Tarski game for the formula 3xiyf{x) < f{y), the Hayashi 
realizer checks all the natural numbers to be sure that an n such that \/yf{n) < f{y) is 
eventually found. On the contrary, our realizer yields a strategy for Eloise which bounds 
the number of backtrackings by /(O), as shown in this paper; moreover, what the strategy 
learns is uniquely determined by interaction with the other player. In this case, the Hayashi 
strategy is the same one suggested by the classical truth of the formula, whereas ours is the 
constructive strategy suggested by its classical proof. 

Since learning based realizers are extracted from proofs in HA+EMi (Heyting Arithmetic 
with excluded middle over existential sentences, see chapter 3), one also has an interpreta- 
tion of classical proofs as strategies with 1-Backtracking. Moreover, studying learning based 
realizers in terms of 1-Backtracking games also sheds light on their behaviour and offers an 
interesting case study in program extraction and interpretation in classical arithmetic. 

In the second part of the chapter, we extend the class of learning based realizers from 
a classical version of Godel's system T to a classical version of VCJ- and define a more gen- 
eral "total recursive learning based realizability" . This step is analogous to the (conceptual, 
rather than chronological) step leading from Kreisel realizability to Kleene realizability: one 
extends the computational power of realizers. We then prove a completeness theorem: for 
every implication-and-negation-free arithmetical formula ^4, if Eloise has recursive winning 
strategy in the 1-Backtracking Tarski game associated to ^, then A is also realizable. 

The plan of the chapter is the following. In section §4.2, we recall the definitions and 
results from chapter 3 that we shall need in the present one. In section §4.3, we prove our 
first main theorem: a realizer of an arithmetical formula embodies a winning strategy in 
its associated 1-Backtracking Tarski game. In section §4.4, we extract realizers from two 
classical proofs and study their behavior as learning strategies. In section §4.5, we define 
an extension of the learning based realizability of chapter 3 and in section §4.6 prove its 
completeness with respect to 1-Backtracking Tarski games. 

4.2. Learning-Based Realizability for the Standard Language of Arithmetic 

In this chapter, we will use a standard language of Arithmetic: the symbols Xp, ct>p will 
not occur in the language of formulas, but only in realizers. We recall the definition and 
results we need here. 
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Definition 4.2.1 (Convergence). Assume that is a w.i. sequence of state con- 

stants, and u,v G Tciass- 

(1) u converges in {sijigN if 3i G N.Vj > i.u[sj] = u[si] in TLcam- 

(2) u converges if u converges in every w.i. sequence of state constants. 

We will make use of the following two theorems of chapter 3. 

Theorem 4.2.1 (Stability Theorem). Assume t £ Tciass is a dosed term of atomic type 
A (A £ {Bool,N,S}j. Then t is convergent. 

Theorem 4.2.2 (Zero Theorem). Let t : S be a closed term of Tciass of state and s 
any state constant. Define, by induction on n, a sequence {snjnGN of state constants such 
that: sq = s and Sn+i = Sn^ t[sn]. Then, there exists an n such that t[sn] = 0. 

We now define a language for Peano Arithmetic and then formulate a realizability 
relation between terms of Tciass and formulas of the language. 

Definition 4.2.2 (The language C of Peano Arithmetic). We define: 

(1) The terms of C are all terms t of Godel's system T, such that t : N and FV{t) C 
{x^, . . . ,x^} for some xi, . . . 

(2) The atomic formulas of C are all terms Qti . . .tn of Godel's system T, for some 
Q : — )■ Bool closed term of T, and some terms ti, . . . , t„ oi C 

(3) The formulas of C are built from atomic formulas of C by the connectives V, A, — >■ 
V, 3 as usual. 

We now define the types realizers as in chapter 3 (we only use a different notation in 
order to avoid confusion in the rest of the chapter). 

Definition 4.2.3 (Types for realizers). For each arithmetical formula A we define 
a type [A] of T by induction on A: [P{ti, . . . ,t„)] = S, [A A B] = [A] x [B], [Ay B] = 
Bool X {[A] X [B]), [A-^ B] = [A] [B], [^xA] = N ^ [A], [3xA] = N x [^] 

We give the simplified notion of learning-based realizability we shall use in the following. 

Definition 4.2.4 (Learning-Based Realizability). Assume s is a state constant, t G 
Tciass is a closed term of state 0, A £ Cis a closed formula, and t : [A]. Let t = ti, . . . , t„ : N. 

(1) t llhs P{t) if and only if t[s] = in TLcarn implies P{t) = True 

(2) t \\\-s AaB if and only if not llh^ A and nit llh^ B 
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(3) t llhs Ay B if and only if either po^l-s] = True in TLcam and pit IW-g A, or 
Pot[s] = False in TLcam and p2t llhs B 

(4) t \\\-s A-)' B a and only if for all u, if u llhs A, then tu llh^ S 

(5) t llhs VxA if and only if for all numerals n, tn llh^ A[n/x] 

(6) t \\\-s 3xA if and only for some numeral n, 7rot[s] = n in TLcam and nit llhg ^[n/x] 
We define t llh A if and only if t llh^ A for all state constants s. 

For the soundness result we shall need this theorem of chapter 3. 

Theorem 4.2.3 . If A is a closed formula of C provable in HA + EMi, then there exists 
t S Tciass such that t llh A. 

4.3. Games, Learning and Realizability 

In this section, we define the abstract notion of game, its 1-Backtracking version and 
Tarski games. We also prove our main theorem, connecting learning based realizability and 
1-Backtracking Tarski games. 

Definition 4.3.1 (Games). We define: 

(1) A game G between two players is a quadruple 

{V,Ei,E2,W) 

where V \s a, set, E2 are subsets of F x 1/ such that Dom{Ei) fi Dom{E2) = 0, 
where Dom{Ei) is the domain of Ei, and is a set of sequences, possibly infinite, 
of elements of V . The elements of V are called positions of the game 5 Ei^ E2 are 
the transition relations respectively for player one and player two: {vi,V2) € Ei 
means that player i can legally move from the position vi to the position V2- 

(2) We define a play to be a walk, possibly infinite, in the graph iy,Ei U £"2), i.e. 
a sequence, possibly void, vi :: V2 ■■ ■■■ ■■ Vn ■■ ■■■ of elements of V such that 
{vi,Vi+i) G ElU E2 for every i. A play of the form vi :: V2 ■■■■■■■ Vn ■ is said 
to start from vi. A play is said to be complete if it is either infinite or is equal 
to vi :: . . . :: Vn and f„ ^ Dom{Ei U £^2)- W is required to be a set of complete 
plays. If p is a complete play and p G W , we say that player one wins in p. If p is 
a complete play and p ^ W , we say that player two wins in p. 

(3) Let Pg be the set of finite plays. Consider a function / : Pq V . A play 

::...:: ti„ :: ... is said to be /-correct if f{vi :: ... :: Vi) = Vi+i for every i 
such that {vi^ViJ^i) G Ei. / is said to be a strategy for player i if for every play 
p = vi Vn such that Vn € Dom{Ei), vi :: . . . :: Vn :: f{p) is a play. 
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(4) A winning strategy from position v for player one is a strategy u : Pg V such 
that every complete cj-correct play v :: vi Vn ■ belongs to W. 

Notation (Concatenation of Sequences). If for i S N, i = 1, . . . , n we have that 
Pi = {Pi)o ■ ■ ■ {Pi)ni is & finite sequence of elements of length n^, with pi :: . . . :: p„ we 
denote the sequence 

(pi)o ::...:: {pi)ni {pk)o {Pk)nk 

where {pi)j denotes the j-th element of the sequence pi. 

Suppose that ai :: a2 :: ... :: o„ is a play of a game G, representing, for some reason, 
a bad situation for player one (for example, in the game of chess, might be a configura- 
tion of the chessboard in which player one has just lost his queen). Then, learnt the lesson, 
player one might wish to erase some of his moves and come back to the time the play was 
just, say, oi, 02 and choose, say, bi in place of 03; in other words, player one might wish to 
backtrack. Then, the game might go on as ai :: 02 :: 61 :: ... b^ and, once again, player 
one might want to backtrack to, say, oi :: 02 :: 61 :: ... :: ftj, with i < m, and so on... As 
there is no learning without remembering, player one must keep in mind the errors made 
during the play. This is the idea of 1-Backtracking games (for more motivations, we refer 
the reader to [9]) and here is our definition. 

Definition 4.3.2 (1-Backtracking Games). Let G = (^,£'1,^2,^^) be a game. 

(1) We define Iback(G) as the game {Pa, E{, E'2,W'), where: 

(2) Pg is the set of finite plays of G 
(3) 

E2 := {{p :: a, p :: a :: b) \ p e Pg,P ■■ a e Pg, {a,b) G E2} 

and 

E[ := {{p :: a, p :: a :: b) \ p e Pg,P :: a € PG,{a,b) £ Ei} U 
{{p :: a:: q, p :: a) \ p,q G Pg,P :: a :: q e PG,a £ Dom{Ei) 
{q = q' --.d^di Dom{E2)),p :: a :: q ^ W}; 
(4) W' is the set of finite complete plays pi :: . . . :: pn of {Pg, E'l, E'2) such that pn G W . 

Note. The pair (p :: a :: g, p :: a) in the definition above of E2 codifies a backtracking 
move by player one (and we point out that q might be the empty sequence). 

Remark. Differently from [9], in which both players are allowed to backtrack, we only 
consider the case in which only player one is supposed do that (as in [27]). It is not that 
our results would not hold: we claim that the proofs in this paper would work just as fine 
for the definition of 1-Backtracking Tarski games given in [9]. However, as noted in [9], any 
player-one recursive winning strategy in our version of the game can be effectively trans- 
formed into a winning strategy for player one in the other version the game. Hence, adding 
backtracking for the second player does not increase the computational challenge for player 
one. Moreover, the notion of winner of the game given in [9] is strictly non constructive 
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and games played by player one with the correct winning strategy may even not terminate. 
Whereas, with our definition, we can formulate our main theorem as a program termination 
result: whatever the strategy chosen by player two, the game terminates with the win of 
player one. This is also the spirit of realizability and hence of this paper: the constructive 
information must be computed in a finite amount of time, not in the limit. 

In the well known Tarski games, there are two players and a formula on the board. The 
second player - usually called Abelard - tries to show that the formula is false, while the 
first player - usually called Eloise - tries to show that it is true. Let us see the definition. 

Definition 4.3.3 (Tarski Games). Let ^ be a closed imphcation and negation free 
arithmetical formula of L. We define the Tarski game for A as the game Tp^ = {V, Ei,E2, W), 
where: 

(1) V is the set of all subformula occurrences of A; that is, V is the smallest set of 
formulas such that, if either AV B or A A B belongs to V, then A,B (^V; if either 
\/xA{x) or 3xA{x) belongs to V, then A{n) € V for all numerals n. 

(2) El is the set of pairs (^1,^2) €VxV such that Ai = 3xA{x) and A2 = A{n), or 
Ai = Ay B and either A2 = A or A2 = B; 

(3) E2 is the set of pairs (Ai, ^2) ^VxV such that Ai = \/xA{x) and A2 = A{n), or 
Ai = AaB and A2 = A or A2 = B; 

(4) W is the set of finite complete plays Ai :: ... :: An such that An = True. 

Note. We stress that Tarski games are defined only for implication-and-negation-free 
arithmetical formulas. Indeed, Iback(T^), when A contains implications, would be much 
more involved and less intuitive (for a definition of Tarski games for every arithmetical 
formula see for example Lorenzen's [18]). 

What we want to show is that if t llh A, then t gives to player one a recursive winning 
strategy in Iback(Tyi). The idea of the proof is the following. Suppose we play as player 
one. Our strategy is relativized to a knowledge state and we start the game by fixing the 
actual state of knowledge as 0. Then we play in the same way as we would do in the Tarski 
game. For example, if there is \/xA{x) on the board and A{n) is chosen by player two, 
we recursively play the strategy given by tn; if there is 3xA{x) on the board, we calculate 
7rot[0] = n and play A{n) and recursively the strategy given by vrit. If there is Avi? on the 
board, we calculate po^i^]; and according as to whether it equals True or False, we play 
the strategy recursively given by pit or p2t. If there is an atomic formula on the board, if it 
is true, we win; otherwise we extend the current state with the state 0iu)t[0], we backtrack 
and play with respect to the new state of knowledge and trying to keep as close as possible 
to the previous game. Eventually, we will reach a state large enough to enable our realizer 
to give always correct answers and we will win. Let us consider first an example and then 
the formal definition of the winning strategy for Eloise. 
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Example (EMi). Given a predicate P of T, and its boolean negation predicate -iP 
(which is representable in T), the realizer Ep oi 

EMi := Vx. 3y y) V \/y^P{x, y) 

is defined as 

Aa'*(XpQ, (0pa, 0), Am" Addpam) 

We now compute a winning strategy for Eloise in the 1-Backtracking game associated to 
EMi. According to the rules of the game Iback(rEivii)! Abelard is the first to move and, for 
some numeral n, chooses the formula 

3y P{n,y)\/\iy^P{n,y) 
Now is the turn of Eloise and she plays the strategy given by the term 

(Xpn, (ct)pn, 0), Am'' Addpnm) 

Hence, she computes Xpn[0] = xp0n = False (by definition 3.2.7), so she plays the 
formula 

yy^P{n,y) 

and Abelard chooses m and plays 

^P{n, m) 

If ^P(n,'m) = True, Eloise wins. Otherwise, she plays the strategy given by 

(Am" Addpnm)m[0] = addp0nm = {(P, n,m)} 
So, the new knowledge state is now {{P, n, m)} and she backtracks to the formula 

3y P{n,y) y\ly^P{n,y) 
Now, by definition 3.2.7, Xpn[{(P, n, m)}] = True and she plays the formula 

3y P{n,y) 

calculates the term 

7ro(<l>pn, 0) [{(P, n, m)}] = ipp{{P, n, m)}n = m 
plays P(n, m) and wins. 

Notation. In the following, we shall denote with upper case letters A, B, C closed arith- 
metical formulas, with lower case letters p, q, r plays of Ta and with upper case letters 
P,Q,R plays of Iback(TA) (and all those letters may be indexed by numbers). To avoid 
confusion with the plays of Ta, plays of lBack(r4) will be denoted as pi, . . . ,pn rather 
than pi :: ... :: p„. Moreover, if P = qi, . . . ,qm, then P,pi, . . . ,pn will denote the sequence 
qi,... ,qm,Pi, ■■■Pn- 

We now define, given a play p of Ta, a term p{p), which we call "the realizer associated 
to p" and which represents the term that should be consulted by Eloise in a position Q,p 
of the game Iback(ryi). 
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Definition 4.3.4 . Fix u such that u llh A. Let p be a finite play of Ta starting with 
A. We define by induction on the length of p a term p(p) G Tciass (read as 'the realizer 
associated to p') in the following way: 



(1) 


Up = 


A, then p{p) 


= u. 








(2) 


Up = 


{q :: 3xB{x) : 


: B{n)) and p{q 


:: 3xB{x)) 


= t, then p{p) 


= TTit 


(3) 


Ep = 


{q :: \/xB[x) : 


: B{n)) and p{q 


:: \/xB[x)) 


= t, then p{p) 


= tn. 


(4) 


Up = 


(qy.BoABi 


:: Bi) and p{q :: 


BoABi) = 


t, then p(j)) = 


■Kit. 


(5) 


Up = 


{q::BiWB2 


:: Bi) and p{q :: 


Bi V B2) = 


t, then p(j)) = 


Pit. 



Given a play P = Q,q :: B of Iback(T^), we set p{P) = p{q :: B). 

A play P of Iback(ryi) may involve a number of backtracking moves by Eloise. In the 
winning strategy we are going to define, each time Eloise backtracks, she must extend the 
current state of knowledge by means of a realizer. In the above definition, we explain how 
Eloise calculates the state associated to P. 

Definition 4.3.5 . Fix u such that u llh A. Let p be as in definition 4.3.4 and P be a 
finite play of lback(T/i) starting with A. We define by induction on the length of P a state 
S(P) (read as 'the state associated to P') in the following way: 

(1) If P = A, then S(P) = 0. 

(2) If P = {Q,p :: B,p::B :: C) and S(Q,p :: B) = s, then S(P) = s. 

(3) If P = {Q,p :: B :: q,p :: B) and S(Q,p :: B :: q) = s and p{Q,p :: B :: q) = t, then 
if t : S, then S(P) = s lyj t[s], else S(P) = s. 

We are now in a position to define a winning strategy for Eloise. Given a play Q,p of 
Iback(r^), she computes the state associated to Q,p and then calls the realizer associated 
to p, which returns to her the next move to be performed. 

Definition 4.3.6 (Winning strategy for Iback(rA)). Fix u such that u llh A. Let p 
and S be respectively as in definitions 4.3.4 and 4.3.5. We define a function uj from the 
set of finite plays of Iback(T^) to set of finite plays of Ta; w is intended to be a recursive 
winning strategy from A for player one in lback(r/i). 

(1) If p{P,q :: 3xB{x)) = t, S(P,g :: 3xB{x)) = s and (7rot)[s] = n, then 

uj{P,q :: 3xB{x)) = q :: 3xB{x) :: B{n) 

(2) If p{P, q :: BV C) =t and S(P, q :: B \J C) = s, then if (poOH = True then 

uj{P,q:: By C) =q:: By C :: B 

else 

uj{P,q:: BVC) = q :: B V C :: C 
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(3) If An is atomic, An = False, p{P, Ai :: ■ ■ ■ :: An) = t and Ai 
then 

uj{P,Ai An) = Ai ■ :: A, 

where i is equal to the smallest j < n such that p{Ai :: • • • :: Aj) 
Aj = 3xC{x) A Aj+i = C{n) A {ttow)[s y t[s]] / n 

or 

Aj = BiV B2A Aj+i = BiA {pow)[s y t[s]] = False 

or 

Aj = BiV B2A vlj+i = ^2 A {pow)[s y = True 
If such j does not exist, we set i = n. 

(4) In the other cases, q) = q. 

Lemma 4.3.1 . Suppose u llh A and p, T,,u} as in definition 4-3.6. Let Q be a finite uj-correct 
play of Iback(TA) starting with A, p{Q) = t, = s. If Q = Q',q' :: B, then t llh^ B. 

Proof. By a straightforward induction on the length of Q. 

(1) If Q = ^, then t = p{Q) = u llh, A. 

(2) If Q = P,g :: 3xB{x),q :: 3xB{x) :: B{n), then let t' = p{P,q :: 3xB{x)). By 
definition of S, s = T,{P,q :: 3xB{x)). Since Q is w-correct and (g :: 3xB{x),q :: 
3xB{x) :: B{n)) G Ei, we have uj{P,q :: 3xB{x)) = q :: 3xB{x) :: B{n) and so 
n = ['KQt')[s\. Moreover, by definition of p, t = irit'; by induction hypothesis, 
t' \\\-s 3xB{x); so, t = -Kit' \\\-s B{n). 

(3) If Q = P,q :: B y C,q :: B y C :: B, then let t' = p{P, q::ByC). By definition of 
S, s = :: B\JC). Since Q is w-correct and (q :: By C,q :: B\/ C :: B) e Ei, 
we have uj{P, q :: B y C) = q :: B y C :: B and so {pQt')[s] = True. Moreover, by 
definition of p, t = pit'; by induction hypothesis, t' 111-, B y C; so, t = pit' llhg 5. 
The other case is analogous. 

(4) If Q = P,g :: yxB{x),q :: VxS(x) :: B{n), then let i' = p{P,q :: VxS(3;)). By 
definition of S, s = T,{P,q :: yxB{x)). By definition of p, t = t'n; by induction 
hypothesis, llh^ \/xB{x)] hence, t = t'n llh^ B{n). 

(5) If Q = P,q :: B AC,q :: B AC :: B, then let f = g :: S A C). By definition 
of S, s = T,{P,q :: B A C). By definition of p, t = TTQt'; by induction hypothesis, 
t' III-5 B AC; hence, t = vrot' llh.; B. The other case is analogous. 

(6) If Q = P, ^1 :: • • • :: An,Ai ::•••:: Ai, i < n, An atomic, then Ai = A. Further- 
more, if Ai :: ■ ■ ■ :: An) = s' and t' = p{P, Ai :: ■ ■ ■ :: An), then s = s' P t'[s']. 
Let tj = p{Ai :: ••• :: Aj), for j = l,...,i. We prove by induction on j that 
tj llhg Aj, and hence the thesis. If j = 1, then ti = p{Ai) = p{A) = u llh^ A = Ai. 
If j > 1, by induction hypothesis t^ IIHg Ak, for every k < j. If either Aj^i = 
\/xC{x) or Aj-i = Co A Ci, then either tj = tj^in and Aj = C{n), or tj = Tr^tj^i 



An) — s, 
= w and either 
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and Aj = Cm- in both cases, we have tj IW-g Aj, since tj-i llhg Aj-i. Therefore, 
by definition of u and i and the uj- correctness of Q, the remaining possibihties 
are that either Aj-i = 3xC{x), Aj = C{n), tj = nitj-i, with (7rotj_i)[s] = n; 
or Aj^i = Ci V C2, Aj = Cm, tj = Pmtj-i and {pQtj-i)[s] = True if and only if 
771 = 1; in both cases, we have tj llh^ Aj. 

□ 

Theorem 4.3.1 (Soundness Theorem). Let A be a closed negation and implication free 
arithmetical formula. Suppose that u llh A and consider the game lback(T/i). Let uj he as 
in definition 4-3.6. Then uj is a recursive winning strategy from A for player one. 

Proof. We begin by showing that there is no infinite oj-correct play. 
Let P = pi, . . . ,pn, . . . be, for the sake of contradiction, an infinite tj-correct play, with 
pi = A. Let Ai Ak he the longest play of such that there exists j such that 

for every n > j, pn is of the form Ai :: • • • :: A^ :: Qn- A\ :: • • • :: A}^ is well defined, 
because: pn is of the form A :: q'^ for every n; the length of pn is at most the degree of the 
formula A; the sequence of maximum length is unique because any two such sequences are 
one the prefix of the other, and therefore are equal. Moreover, let {njjjgN be the infinite 
increasing sequence of all indexes Ui such that Pm is of the form Ai :: • • • :: A^ :: Qm and 
Pm+i = Ai :: • • • Ai^. (indeed, {njjjgN must be infinite: if it were not so, then there would 
be an index / such that for every n > j' , Pn = Ai :: - - - Af^ :: A^+i :: g, violating the 
assumption on the maximal length Ai :: • • • :: A^). Aj^, if not atomic, is a disjunction or 
an existential statement. 

Let now Sj = S(pi, . . . and t = p{Ai :: ■ ■ ■ :: ^4^). For every i, Si < Sj+i, by definition of 
S. There are three cases: 

1) Ak = 3xB{x). Then, by the Stability Theorem (Theorem 4.2.1), there exists m such 
that for every a, if > m, then (vrot)[sna] = i'^ot)[sm]- Let 

h := (7rot)[s„„ y ti[s„J] = (7rot)[s„„+i] 

where ti = p{pi, . . . ,Pna)- So let a be such that > m; then 

Pna+2 = ^{Pi,-- - ,Vna+i) = ^{Pi, Ai Ak) = Ai Ak B{h) 

Moreover, by hypothesis, and since Pua+i = Ai A^., we have 

^^(,+1) = Ai Ak :: 9n(,+i) = Ai A^ :: B{h) :: q' 

for some q' and = Ai :: • • • :: A^: contradiction, since 

h = (vrot)[s„^+i] = (vrot)[s„(^_^i)+l] = {lTot)[Sn^^^,^ ^ HSni^a+i)]] 

where ^2 = p{pi, - - - ,Pn(,+i)), whilst h / (7rot)[sn(,+i) liJJt2[sn(,+i)]] should hold, by definition 
of UJ (point (3)). 

2) = i? V C. This case is totally analogous to the preceding. 

3) is atomic. Then, for every n > j , pn = Ai :: - - - :: Af^.. So, for every n > j, 
Sn+i = Sn lyj t[sn] and hence, by Theorem 4.2.2 there exists m > j such that t[sm] = 0- 
But t W^Sm by Lemma 4.3.1; hence, Aj. must equal True, and so it is impossible that 
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{Pm,Pm+i) = (^1 :: • • • :: Ak,Ai Ak) € E[: contradiction. 

Let now p = pi, . . . ,pn he a complete finite Li;-correct play. p„ must equal Bi :: ■ ■ ■ :: B^, 
with Bk atomic and B]^ = True: otherwise, p wouldn't be complete, since player one would 
lose the play pn in T4 and hence would be allowed to backtrack by definition 4.3.2. 

□ 

4.4. Examples 

In this section we include two natural deduction classical proofs of two simple combi- 
natorial statements, using only Excluded Middle for semi-decidable statements, then we 
extract a constructive content using our realizability semantics. For each of them, we in- 
terpret the program we shall extract using our game interpretation of the learning based 
realizability semantics. 

4.4.1. Minimum Principle for Functions over Natural Numbers. The mini- 
mum principle states that every function / over natural numbers has a minimum value, i.e. 
there exists a f{n) € N such that for every m € N /(m) > /(n). We can prove this principle 
in HA + EMi, for any / in the language. We assume P{y,x) = f{x) < y, but, in order to 
enhance readability, we will write f{x) < y rather than the obscure P{y,x). We define: 
Lessef{n) := 3af{a) < n 
Lessf{n) := 3af{a) < n 
Notlessf{n) := Va/(a) > n 

Then we formulate - in equivalent form - the minimum principle as: 

Hasminf := 3y. Notlessf{y) A Lessef{y) 

The informal argument goes as follows. We prove by induction on n that for every k, if 
f{k) < n, then / has minimum value. If n = 0, we just observe that f{k) < 0, implies f{k) 
is the minimum value of /. Suppose now n > 0. If Notlessf{f{k)) holds true, we are done, 
f{k) is the minimum of /. Otherwise, Lessf{f{k)) holds, and hence f{a) < f{k) < n for 
some a given by an oracle. Hence f{a) < f{k) — 1 < n — 1 and we conclude that / has a 
minimum value by induction hypothesis. 

Now we give the formal proofs, which are natural deduction trees, decorated with terms 
of Tciass, as formalized in chapter 3. We first prove that Vn. {Lessef{n) — > Hasminf) — > 
{Lessef{S{n)) — ?• Hasminf) holds. 

[Notlessf{S{n))] [Less f (Sin))] 
Ep : Vn. Notlessf{S{n)) V Lessf{S{n)) Ti T2 

Epn:Notlessf{S{n))\/Lessf{S{n)) Hasminf Hasminf 

D : Hasminf 
XW2D : Lessef [S{n)) — >■ Hasminf 
XW1XW2D : (Lessef (n) — >■ Hasminf) — >■ {Lessef {S{n)) — > Hasminf) 
\n\w1\w2D : yn(Lessef{n) — > Hasminf) (Lessef {S{n) — > Hasminf) 

where for lack of space the term D is defined later, Ti is the tree 

Vi : Notlessf(S(n)) W2 ■ Lessef (S(n)) 
{vi,W2) ■ Notlessf(S(n)) A Lessef (S(n)) 
(S(n), {vi,W2)) ■ Hasminf 

and T2 is the tree 
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[X2 : f{z) < S(n)] 
X2 ■■ f{z) < n 

wi : [Lessef{n) — > Hasminf] {z,X2) '■ Lessef{n) 
V2 ■ [Lessf{S{n))] wi{z,X2) ■ Hasminf 

u'i(7roW2, 7''i'^2) '■ Hasminf 

We prove now that Lessef(O) Hasminf 

xi : [f{z) < 0] 



xi : f{z) = 



XI : /(a) > f{z) : /(z) < f(z) 



Xaxi : Notlessf(f{z)) {z, 0) : Lessef{f{z)) 
{\axi,{z, 0)} : Notlessf{f{z)) A Lessef{f{z)) 
w : [Lessef{0)] {f{z), {Xaxi, {z, 0))) : Hasminf 

(/(ttqw), (AaTTiw, (ttqw, 0))) : Hasminf 
F Xw{f{T:ow), (Xaniw^ (ttow, 0))) : Lessef{0) — Hasminf 

Therefore we can conclude with the induction rule that 

Xa^ RF{XnXwiX'W2D)a : \/x.Lessef{x) Hasminf 

And now the thesis: 

: /(O) < /(O) Xa^ RF{XnXwiXw2D)a : \lx.Lessef{x) Hasminf 



(0, 0) : Lessef{f{0)) RF{XnXwiXw2D)f{0) : Lessef{f{0)) Hasminf 
M -.^ RF{XnXwiXw2D)f{O){O,0) : Hasminf 

Let us now define 

D := if XpS(n) then wi{<^pS{n), 0) else (S(n), (A/3 (Addp)S(n)/3, 1^2)) 

Let s be a state and let us consider M, the realizer of Hasminf , in the base case of the recur- 
sion and after in its general form during the computation: RF{XnX'WiXw2D)f{O){m,0)[s]. 
If /(O) = 0, 

M[s] = RF{XnXwiXw2D)f{O){O,0)[s] = 
= F(O,0) = (/(O),(Aa0,(O,0)) 
If /(O) = S(n), we have two other cases. If XpsS{n) = True, then 

RF{XnXwiXw2D)S{n){m,0)[s] = 
= {XnXwiXw2D)n{RF{XnXwiXw2D)n){m, 0)[s] = 
= RF{XnXwiXw2D)n{(i>p{Sin)),0)[s] 

If XpsS{n) = False, then 

RF{XnXwiXw2D)S{n){m,0)[s] = 
= {XnXwiXw2D)n{RF{XnXwiXw2D)n){m, 0)[s] = 
= (S(n),(A/3 (addp)sS(n)/3,(m,0))) 
In the first case, the minimum value of / has been found. In the second case, the operator 
R, starting from S(n), recursively calls itself on n; in the third case, it reduces to its nor- 
mal form. From these equations, we easily deduce the behavior of the realizer of Hasminf . 
In a pseudo imperative programming language, for the witness of Hasminf we would write: 



n := /(O); 
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while {xpsn = True, i.e. 3m such that f{m) < n E s) 
do n := n — 1; 
return n; 

Hence, when /(O) > 0, we have, for some numeral k 

M[s] = {k, (A/5 (addp)sfc/3, {(ppsk, 25))) 

It is clear that k is the minimum value of /, according to the partial information provided by 
s about /, and that f{ippsk) < k. If s is sufficiently complete, then k is the true minimum 
of/. 

The normal form of the realizer M of Hasminf is so simple that we can immediately extract 
the winning strategy cu for the 1-Backtraking version of the Tarski game for Hasminf . 
Suppose the current state of the game is s. If /(O) = 0, Eloise chooses the formula 

Notlessf{0) A LessefiO) 

and wins. If /(O) > 0, she chooses, for k defined as above, 

Notlessf{k) A Lessef{k) = Va f{a) >kA3a f{a) < k 

If Abelard chooses 3a f{a) < k, she wins, because she responds with f{ippsk) < k, which 
holds. Suppose hence Abelard chooses 

Va f{a) > k 

and then > k. If it holds, Eloise wins. Otherwise, she adds to the current state s 

(A/3 iaddp)sk(3)(3 = (addp)sA;/3 = {/(/3) < k} 
and backtracks to Hasminf and then plays again. This time, she chooses 

Notlessf{f{f^))ALessef{fm 

(using /(/?), which was Abelard's counterexample to the minimality of k and is smaller than 
her previous choice for the minimum value). After at most /(O) backtrackings, she wins. 

4.4.2. Coquand's Example. We investigate now an example - due to Coquand - 
in our framework of realizability. We want to prove that for every function over natural 
numbers and for every a € N there exists x G N such that /(x) < f{x + a). Thanks to the 
minimum principle, we can give a very easy classical proof: 

[Notlessf{fj,) A Lessef{fj,)] 
Notlessf{fj.) 

f{z + a)>tJi [/(z) < 

f{z)<f{z + a) 

[Notlessfdj.) A Lessefdi)] 3xf{x) < f{x + a) 

Lessef{fj,) \ia3xf{x) < fix + a) 

Hasminf \/a3xf{x) < f{x + a) 

\/a3xf{x) < f{x + a) 

The extracted realizer is 

Aa(7ro7ri7riM, ttqttiM {-KQiriTriM + a) ItU vriTTivri/i) 
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where M is the reahzer of Hasminf . m := 7ro7ri7riM[s] is a point the purported minimum 
value fi := ttqM of / is attained at, accordingly to the information in the state s (i.e. 
/(m) < /i). So, if Abelard chooses 

3x f{x) < f{x + a) 

Eloise chooses 

f{m) < f{m + o) 

We have to consider the term 

U[s\ := 7ro7riM(7ro7ri7riAf + a) W 7ri7ri7riAf[s] 

which updates the current state s. Surely, 7ri7ri7riM[s] = 0. 7ro7riM[s] is equal either to 
A/3 (addp)sfiP or to Aa0. So, what does U[s] actually do? We have: 

U[s] = TTQiTiAd {TTQiriTTiM + a)[s] = 7ro7riM(m + a)[s] 

with either 7rovriM(m + a)[s] = or 

7ro7riM(m + a)[s] = {f{m + a) < f{m)} 

So U[s] tests if f{m + a) < f{m); if it is not the case, Eloise wins, otherwise she enlarges the 
state s, including the information f{m + a) < f{m) and backtracks to 3xf{x) < f{x + a). 
Starting from the state 0, after k + 1 backtrackings, it will be reached a state s' , which will 
be of the form {f{{k + l)a) < f{ka), . . . ,/(2a) < f{a),f{a) < /(O)} and Eloise will play 
f{{k + l)a) < f{{k + l)a + a). Hence, the extracted algorithm for Eloise's witness is the 
following: 

n := 0; 

while /(n) > f(n + a) 
do n := n + a; 
return n; 

4.5. Total Recursive Learning-Based Realizability 

The realizability notion introduced in definition 4.2.4 is very interesting from the con- 
structive point of view. But precisely for that reason, the system Tciass fails to realize every 
formula for which an Eloise recursive winning strategy exists in its associated 1-Backtracking 
Tarski game, as the following theorem implies: 

Theorem 4.5.1 (Incompleteness of Tclass)- There is a arithmetical sentence A such 
that Eloise has recursive winning strategy in Iback(T^), but no term of system Tciass realizes 
A. 

Proof. Take any total recursive function / : N — ?• N not representable by any term of 
system T of type N ^ N. Let n be the code of / in the enumeration of Turing machines 
assumed by Kleene's primitive recursive predicate Txyz. Then the formula A := WyBzTnyz 
asserts the totality of / and hence it is true. Clearly, Eloise has a winning recursive strategy 
in Iback(TA): for any y, she may backtrack until she finds an m such that Tnym holds. 
Suppose, along the way of contradiction, that for some t of system Tciass, t HI" A. Then, as 
proven in chapter 5, there exists a term « : N ^ N of system T such that, for every numeral 
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I, Tnl{ul) holds. Prom this, it easily follows that / can be coded by a term of system T, 
which is a contradiction. 

□ 

The only way around this issue, which is the purpose of this section, is to extend our 
notion of realizability and increase the computational power of our realizers, in order to be 
able to represent any partial recursive function and in particular every recursive strategy of 
1-Backtracking Tarski games. So, we choose to add to our calculus a fixed point combinator 
Y, such that for every term u : A ^ A, Yu = u{Yu), getting the full power of VCT (see for 
example Gunter [25]). 

Definition 4.5.1 (Systems PCT'class and PC-Flearn )• We define VCTc\a,ss and PCJ"Lcarn 
to be, respectively, the extensions of Tciass and TLoam obtained by adding for every type A 
a constant of type {A ^ A) ^ A and a new equality axiom Y fi,u = u{Yf^u) for every 
term u : A —?■ A. 

Since in VCFc\e,ss there is a schema for unbounded iteration, properties like convergence 
do not hold anymore, for terms may even not have a normal form. So we have to ask our 
realizers to be convergent. Hence, for each type A of VCJ-'ciass we define a set ||^|| of terms 
u : A which we call the set of stable terms of type A. We define stable terms by lifting the 
notion of convergence from atomic types to arrow and product types. 

Definition 4.5.2 (Convergence for 'PCJ'class)- Assume that {sjjigN is a w.i. sequence 
of state constants, and u,v £ VCTcinss- 

(1) u converges in {sijjgN if there exists a normal form v such that 3iVj > = v 

in PCJ"Lcarn- 

(2) u converges if u converges in every w.i. sequence of state constants. 

Definition 4.5.3 (Stable Terms). Let {sijigN be a w.i. chain of states and s € S. 
Assume A is a type. We define a set ||^|| of terms t G VCFquss of type A^ by induction on 
A. 



(1) 


||S|| = {t 


S t converges} 




(2) 


||N|| = {t 


N t converges} 




(3) 


Bool = 


{t : Bool t converges} 




(4) 


\\A X B\\ 


= {t : Ax B \ TTQt G A ,7rit G 


\B 


(5) 


\\A B\\ 


= {t : A^ B \ \lu \\A\\,tu £ 


\B 



If t G ll^lli we say that f is a stable term of type A. 

Now we extend the notion of realizability with respect to VCFcuss and 'PCJ^Learn- 
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Definition 4.5.4 (Total Recursive Learning-Based Realizability) . Assume s is 
a state constant, t e VCFcuss is a closed term of state 0, A ^ C \s & closed formula, and 
t £ \\[A]\\. Let t = ti,...,tn : N. 

(1) t \\\-s P{t) if and only if t[s] = in VCJ-i^pg^y-^ implies P{i) = True 

(2) t \\\-s AaB a and only if not llh^ A and nit llh^ B 

(3) t llhs A V -B if and only if either pot[s] = True in VCJ^i^eum and pit llhg A, or 
Po^N = False in PCJ^Lcam and B 

(4) t llhs ^ ^ B if and only if for all u, if u llh^ A, then tu llh^ S 

(5) t \\\-s VxA if and only if for all numerals n, tn llhg A[n/x] 

(6) t llhs if and only for some numeral n, 7rot[s] = n in PCJ^Lcam and vrit llhcj 74[n/3;] 
We define t llh A if and only if t llh^ A for all state constants s. 

We observe that theorem 4.2.2 holds as well for the stable terms of VCJ-Qiass, for it is 
a consequence of the Stability theorem 4.2. L Hence, the Soundness theorem 4.3.1, which 
depends only on the definition of realizability, stability and theorem 4.2.2, also holds for 
realizers of VCFc\a.sa- That is, we have 

Theorem 4.5.2 (Soundness Theorem (PCJ"class))- Let A he a closed negation-and- 
implication free arithmetical formula. Suppose that u G VCJ- ciass o,nd u llh A and consider 
the game Iback(Tyi). Let co be as in definition 4-3.6. Then u is a recursive winning strategy 
from A for player one. 

4.6. Completeness 

4.6.1. Idea of the Proof. In this section, we prove our completeness theorem: if an 
implication-and-negation-free arithmetical formula has a winning recursive strategy in its 
associated 1-Backtracking Tarski game, then it is realizable by a term of VCFcuss- 

The idea of the proof follows naturally from the very meaning of learning based re- 
alizability. In order to realize a formula, one has to provide in the first place a Kleene- 
Kreisel-style realizer of the formula, recursive in an oracle for the Halting problem. This 
corresponds to the fact that the terms of Tciass contain symbols for non computable func- 
tions which are in the same Turing degree of the aforementioned oracle. That is why one can 
see learning based realizability as a way of "programming with non computable functions" . 
Hence, one would like to apply directly Berardi et al. [9] result: given an implication-and- 
negation-free arithmetical formula, if there exists a recursive winning strategy for Eloise 
in its associated 1-Backtracking Tarski game, then there also exists a winning strategy for 
Eloise in its associated Tarski game, recursive in an oracle for the Halting problem. 

However, that result is not enough for our purposes. According to learning based re- 
alizability, together with an oracle-equipped Kleene-Kreisel-style realizer, one has also to 
provide an effective method for learning oracle values in a convergent way and show that 
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the realizer is always defined, whatever oracle approximations are used. Hence, we have to 
refine Berardi et al. result and prove that oracle values can be learned by counterexamples 
and the program is not "perturbed" by the oracle approximations used. More precisely, we 
show that the inefi'ective oracle strategy given in [9] can be made more effective by using the 
novel ideas of learning based realizability: we first approximate the strategy by allowing it 
to use in the computations only approximated oracles; then we show that good enough ap- 
proximations can be attained by a process of intelligent learning by counterexamples. One 
this task will be accomplished, the completeness theorem will follow just by formalizing the 
argument. 

We now give an informal overview of the construction to be carried out. This should 
serve the reader as a guide to the next technical sections. Suppose that w is a recursive 
winning strategy for Eloise in Iback(T^). We start by describing a winning strategy for 
Eloise in Ta-, which is recursive in some oracle for the Halting problem. We begin with 
some terminology. 

Definition 4.6.1 (Improvable, Optimal Plays). We say that a w-correct play 

Q,Aq:: ... :: Ai 

of Iback(T^), with Ai of the form 3xB ox By C, is improvable if there exists a w-correct 
play of Iback(T^) of the form 

Q,Aq:: ... :: Ai,Q\AQ Ai 

and we call this latter play an improvement of the former. Moreover, a play is said to be 
optimal if it is not improvable. 

The reason why a play 

Q,Aq:: ... :: Ai,Q\AQ Ai 

is called an improvement of Q,Aq :: ... :: Ai is that the former gives more information to 
the strategy a; in order to choose the next move for Eloise. Moreover, if Q,Aq::... :: Ai is 
optimal, whatever cj-correct continuation of the game we may consider, uj will not backtrack 
to Aq Ai anymore. Since any such continuation will extend the play 

Q,Aq:: ... :: Ai^Ao Ai :: Ai+i 

where 

uj{Q, Aq :: . . . :: Ai) = Aq A^ :: Ai+i 

the choice of ^i+i operated by uj is the best possible. 

The oracle X^; that we consider answers to questions of the form: is the play Q,Aq :: 
. . . :: Ai improvable? To facilitate computations we also consider an oracle <t> e which given 
the code of a play Q, Aq :: ... :: Ai returns the code of a play 

Q,Ao:: ... :: Ai,Q',Ao Ai 

whenever Q,Aq :: ... :: Ai is improvable and returns a dummy code otherwise. Observe 
that the two oracles X^; and <t>£; are of the same Turing degree, so exactly one of them 
would suffice. Furthermore, observe that one can define a program taking as input a code 
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of an improvable play Q, Aq Ai and returning the code of an optimal improvement 

of the form 

Q,Ao:: ... :: Ai,Q',Ao Ai 

(just iterate <^e)- 

Suppose now that Aq :: ... :: 3xB is a position in the game Ta- How should Eloise 
move? The idea, coming from [9], is to compute, using our oracles, an optimal play of 
lback(T4) of the form 

Q,Ao :: ... :: 3xB 

Then, Eloise should respond by first computing 

U}{Q, Aq :: . . . :: 3xB) = Aq :: . . . :: 3xB :: B{n) 

and then choosing the formula B{n). The idea is that B{n) is a good choice, since no 
backtracking to ylo - • • • - 3x^1 is ever to be done, following the strategy u. 

More precisely, Eloise, while playing, simultaneously constructs a sequence of plays 
Qo) ■ ■ ■ ,Qk of lback(T/i) in the following way. She first defines Qo to be an optimal im- 
provement of A. Then, suppose k > and that she is in the position 

:: ... :: Ak 

of the game Ta and has constructed a sequence of plays Qq, . . . ,Qk of lback(r4) such that 

i) each play Qi is of the form Q'^jAq :: ... :: A^; 

ii) For all i < k, Qj+i extends Qi] 

iii) For all i, Qi is optimal; 

iv) For all i, Qi is w-correct. 

Then if Abelard has to move and chooses A^+i, Eloise defines Qk+i as an optimal 
improvement of 

Qk,AQ ::...:: Aj. :: A^j^i 
which she can compute using the oracles. If Eloise has to move, she computes 

uj{Qk) = Ao :: . . . :: Ak :: ^fc+i 

she chooses as next move A^+i and she defines Qk+i as an optimal improvement of 

Qk,AQ ::...:: Aj. :: A^+i 

which she can again compute using the oracles. It is clear that the sequence Qi, • • • ,Qk+i 
still satisfies all properties i)-iv). 

Suppose now that a complete play ■■■ of has been played by Eloise 

following the above strategy and suppose by contradiction that A^ = False. Then, since 
by iv) Qn is w-correct and a; is winning, we have 

Uj{Qn) = Aq Ai 

for some i < n, which represents a backtracking move performed by u. Since by ii) Qn 
extends Qi, we have that Qn, Aq Ai can be written as 

Qi, Q, Aq :: ... :: Ai 
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for some Q. As a consequence, Qi = Q[,Aq :: . . . Ai can be improved, which contradicts the 
optimahty of Qi stated at point iii). 

Thus we have a winning strategy for Eloise, recursive in the oracles Xe,'^e- Now, 
however, we want Eloise to follow the very same strategy, but using approximations of 
those oracles in the place of the original ones. Of course, responses of approximated oracles 
are not to be always trustable. However, we will prove that correct oracle values can be 
learned by counterexamples and therefore that the use of the oracles may be replaced by a 
learning mechanism. According to learning based realizability, we will have in particular to 
prove that whenever the "approximated" strategy does not lead Eloise to win Ta, then some 
new value of the oracles can be learned: at least, from a failure Eloise corrects something 
old and gains something new, which is a perfect example of a self- correcting strategy. 

Now, suppose that a complete play Aq :: ... :: An of Ta has been played by Eloise 
following the new "approximated" strategy. We still may suppose that Qo, . . . ,Qn satisfy 
i), ii), iv). However, they satisfy only the following weaker 

iii)' For all i, Qi is optimal, according to the response of the current oracle approxima- 
tions. 

In other words, it might happen that our approximated oracles believe Qi to be not im- 
provable, whilst Qi actually is. Since iii) does not hold any more, the previous argument 
- that has shown An = True - now fails and it might still occur that An = False. How 
Eloise is to learn from this counterexample? She computes u}{Qn), which is of the form 
Aq :: . . . :: Ai, since it represents a backtracking move performed by u. Then as before she 
writes Qn, Aq :: . . . :: Ai as 

Qi,AQ Ai,Q,AQ Ai 

As a consequence, she finds out that Q'i,AQ :: . . . Ai can be extended as to contradict the 
approximated-oracle prediction of point iii)' and hence collects a new value of the oracle. 

In the next two sections, we spell out the details of the construction and prove that the 
above Eloise strategy is sound and in fact convergent. In order to enhance readability and 
separate the important ideas from technicalities, we split the construction into two parts. 
First, we define the concept of learning strategy, which represents a translation of our 
realizability notion into the language of game theory, and show that any winning strategy 
in Iback(ryi) can be transformed into a learning strategy in Ta. Secondly, we show that in 
fact any learning strategy in Ta can be translated into a learning based realizer of A. 

4.6.2. Winning 1-Backtracking Strategies into Learning Strategies. For the 

rest of the paper, fix a closed implication-and-negation-free arithmetical formula A and let 
Ta be its associated Tarski game. Fix moreover a primitive recursive enumeration of the 
plays of Iback(r^) and let uj he a winning recursive strategy from A for player one in the 
game Iback(Tyi). We assume, without loss of generality, that lo performs backtracking moves 
only in front of atomic formulas; that is, we assume that for every play Q, Aq :: . . . :: An, if 

uj{Q,Aq An) = Aq . :: Ai 

then An is atomic. Clearly, any winning strategy can be transformed accordingly to this 
requirement: any backtracking move can be delayed by dummy moves and be performed in 
front of an atomic formula. 
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First of all, we formalize the coding of plays into numbers which has to be represented 
in our calculus. 

Definition 4.6.2 (Abstract Plays of Ta, Coding terms). A sequence of arithmeti- 
cal formulas Aq :: ... :: An is said to be an abstract play of T^, if Aq = A and for all i 

i) if Ai = MxB or Ai = 3xB, then Aj+i = B; 

ii) if Ai = BqA Bi or A, = BqM Bi, then ^j+i = Bq or AiJ^i = Bi. 

Let moreover p = Aq A/, he any abstract play of Ta- By \p\ : N, we denote a 

term of VCJ- ciass having as free variables precisely the variables occuring free in the for- 
mulas of p and such that, for every sequence of numerals n and sequence of variables x 
comprising all the free variables of p, \p\[n/x] is equal to the numeric code of the play 
q = Ao[n/x] ::...:: Ak[n/x]. 

□ 

We define now a translation of learning based realizability into the language of Tarski 
games. The translation consists of a pair {go,gi) of terms of VCJ-c\ass'- the first one describes 
a strategy for Eloise in Ta, and the second one decides when Eloise should backtrack. 

Definition 4.6.3 (Learning Strategy). Let Ta = {V,Ei,E2,W). Let g = {go,gi) be 
a pair of terms of VCJ- c\&ss respectively of types N ^ N and N ^ S. 

For every state s, we say that a play Aq :: . . . :: An of Ta is g\s\-correct if for every i < n 
such that {Ai,Ai^i) G Ei 

go[s]{\Ao Ai\) = \Ai+i\ 

holds. 

g is said to be a learning strategy from A if it satisfies the following conditions: 

(1) (Soundness) For every state s and play p = A :: Aq :: . . . :: An such that 
An G Dom{Ei), if 5o[s](bl) = |^n+i|, then ^ :: - ••• - An :: An+i is a 
play. 

(2) (Convergence) For every play p starting from A, go{\p\) and 5i(|p|) converge. 

(3) (Learning) For every state s and complete (7[s]-correct play p starting from A, if 

gi[s]{\p\) = ^ pGW 

We observe that conditions (2) and (3) of definition 4.6.3 correspond respectively to the 
convergence property that VCJ- cuss realizers must have and to the learning condition in 
realizability for atomic formulas. 

We now define a predicate : — )> Bool of T, which codes the improvement relation 
between plays of Iback(r^) we are interested in. In the following, our terms of 'PCJ-'ciass will 
make use only of the oracle constants X^; and (JJ^;, which are in the syntax of Tciass (recall 
chapter 3, definition 3.2.6) and hence of VCFc\e,ss- Moreover, we define a term ^' : N — )• N, 
which will be our fundamental computational engine. Given a code of a w-correct play, ^ 
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is intended to return the code itself or the code of an optimal improvement, according to 
the oracles Xe,'^e- ^' is in general non computable and by using ^ as it is, we could only 
write strategies recursive in the oracles Xe,'^e, as in [9]. Therefore, we shall compute its 
approximations ^[s], by which we will be able to write the "approximated" strategy for 
Eloise we have discussed in section 4.6.1. 

Definition 4.6.4 (Improvement Relation, Optimality Operator '^). Let E ■.n'^ ^ 
Bool a predicate of Godel's T such that Enm = True iff n and m are numerals coding 
respectively (^-correct plays P,p :: A and P,p :: A, Q,p :: A of lback(r/i), with A = 3xB{x) 
OT A = Boy Bi. 

We want to define now a term 'I' : N — > N of VCJ-c\a&& such that the following equation is 
provable in the equational theory of VCTciass ■ 

= if Xez then ^{<i>Ez) else z 

In order to do that, it is enough to let 

a ■= A/^^Az^if Xez then y{<t>Ez) else z 

and take ^ := Y(a). 

The term ^, given a number n, checks whether Xetl = True, i.e. whether there exists a 
number m such that Enm = True. In that case, it computes such an m by calling 0_Bn, and 
continues the computation by calling itself on m; otherwise, it returns n. The termination 
of '^n is guaranteed by the fact that there are no infinite cj-correct plays and each recursive 
call made by ^ extends a current w-correct play (see [9]). li Xe,'^e are interpreted as 
oracles, returns an optimal improvement of the play coded by n. But when X^;, <t>E are 
approximated through a particular state s, in general will return only an improvement 

of the play coded by n, or even n itself. 

We now have to prove the crucial property that for any finite approximation of the oracles 
Xet'^'e - that is, for any state s - the term \I'n[s] has a normal form and that converge: 
^' is "stable" with respect to oracle approximations. 

Proposition 4.6.1 (Stability of ^f). ^ e ||N N|| 

Proof. Let {sjligN be a w.i. chain of states. By definition 4.5.3, we have to prove 
that, for every term t G ||N||, converges. Since t converges to a numeral, it is enough to 
show that for every numeral n, converges. First of all, we observe that for any state s, 
XEsm is equal to True only for a finite number of arguments m. Moreover, by definition 
4.6.4 

"^n[s] = if XEsn then '^{(pEsn) else n 

Hence, by direct computation it can be seen that, for every i G N, ^'n[sj] has a normal form 
and it is equal, for some A; € N, to {LpESi)^n, having defined by induction 

{(fESifn := n, {ipESi)'^^^n := (pESii{(pESi)"^n) 



Moreover, for every m < k 

XESi{{fESiTn) = True (4.1) 
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does hold and hence {(pESi)^'^^n codes a play properly extending the play coded by {ipESi)"^n. 
Now, if did not converge, then there would be two increasing infinite sequences of num- 
bers ko, ki, k2, ■ ■ ■ and mQ,mi,m2, ■ ■ ■ such that, for every i G N, ^'n[smj = {y^ESmJ'''n. 
Furthermore, since Srm < Srm+i and, by (4.1), for every m < ki 

{E, i^E ''mi 
we have that for every m < ki 

as it can be seen by induction on m. Hence, for every z, letting a = ki^i — ki 

{^ESm,+^f'+^n = {ipESm+lfii^ESm.+^f'n) = {^ESm+lf {{'^ ESrmf' u) 

would be the code of a (^-correct play of Iback(Tyi) properly extending the play coded by 
{ipESmi)^^n. Therefore, it would exist an infinite w-correct play of lback(r4), which is 
impossible since w is a winning strategy by hypothesis. 

□ 

We are going to define three terms A, 11, that will implement the learning strategy 
for Eloise in Ta sketched in section 4.6.1. For the sake of readability, we will describe only 
the properties that such terms must satisfy, without explicitly write down those terms. It 
is trivial, however, to actually code our definitions in VCJ-c\a.ss- 

We start by defining a term A : N — )■ S, which is supposed to code the function gi of 
definition 4.6.3. A[s] takes the code of a play of Iback(T^) and builds a state containing 
values of the oracles X^; and '^e that can be drawn from the input play and are not already 
in s. 

Definition 4.6.5 (Learning Term). Let A : N ^> S be a term of VCTcuss described as 
follows. A takes as argument a numeral m. Then, it checks whether m codes a w-correct play 
Q = Aq . . . :: An of lback(r/i), with Aq :: . . . :: An complete play of and An = False. 
If not, it returns a dummy state: 0. Otherwise, it computes uj{Q) = Aq :: ... :: Ai, and, 
for any state s, returns A[s]m, which equals the state containing all the triples {E,nQ,ni) 
not belonging to s and such that 

i) no codes a play Qo, ^0 - • • • - 

ii) ni codes a play Qo, Aq :: . . . :: Ai, Qi, Aq :: . . . :: Ai; 

iii) Qo,Ao Ai,Qi,Ao Ai = Q, Aq Ai. 

Note. The term A must make use of the constant Add^; to create its output A[s]m, 
which is a state, and that is why the triples {E, no, ni) of the above definition 4.6.5 are not 
in s. 

Recall subsection 4.6.1. We have explained that Eloise, while playing in Ta and in 
position Aq :: ... :: Ai, simultaneously constructs a sequence of plays Qo, ■ ■ ■ ,Qi satisfying 
some properties i)-iv). We now define a term 11 : N ^ N of VCTc\ass, which will be used 
to construct that sequence. In particular, 11 takes the code of a play p = Aq Ai 
and yields the code of a play Qi = Q,p of Iback(TA). Again, if 11 is interpreted as a 
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program recursive in the oracles XB,<t>E, it will yield an optimal play Q,p. But when Il[s] 
is computed, the result Q,p may not be optimal because X^;, <i>E have been approximated 
through the state s. 

Definition 4.6.6 (Sequence Constructor n). We describe the behaviour of a term 
n : N — )• N of VCFcxassi intended to take the code \p\ of a play p of Ta and return the code 
|Q,p| of a play Q,p of Iback(r^). The definition of n[s](|p|) runs by induction over the 
length of p and is distinguished by cases: 

(1) lip = A, then Ii[s\{\p\) = ■^[s\{\A\). 

(2) lip={q :: B) and 

Il[s]{\q\) = \Q,q\ 

then 

n[s](b|) = *[s](|Q,g,g::i?|) 

□ 

We now prove the convergence of A and 11. 
Proposition 4.6.2 (Stability of A and IT). A g ||N S|| and n G ||N N||. 

Proof. Again, to prove that A € ||N — )■ SH it is enough to show that for every numeral 
n. An converges. Let then {sjjjgN be a w.i. chain of states. By definition 4.6.5, whatever 
Si is, A[sj]n construct a finite set of triples {E,nQ,ni), which depends only on n, and then 
decides to output some of the triples, according as to whether they are in Sj or not. This 
determination stabilizes for large enough Sm', that is, for all m' > m, A[sm']n = A[sm]- 

The convergence of Iln follows by straightforward induction on the length of the play 
coded by n and by the convergence of ^. 

□ 

We now put together the terms ^, A, 11 in order to define a learning strategy for Eloise 
in Ta- 

Definition 4.6.7 (Learning Strategy for Ta). We describe the behavior of a pair of 
terms 0, = {Qq, Oi) respectively of type N — > N and N — )■ S of VCJ-'ciass, intended to represent 
a learning strategy for Ta- The definition of J7j[s](|p|) is distinguished by cases: 

(1) p = q :: 3xB{x) and 

U[s]i\q :: 3xB{x)\) = \Q,q:: 3xB{x)\ 

and 

uj{Q,q:: 3xB{x)) = q :: 3xB{x) :: B{n) 
then no[s]{\p\) = \B{n)\. 

(2) Up = q:: BoV Bi and 

U[s]i\q ■■ BoV Bi\) = \Q,q :: BoV Bi\ 
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and 

uj{Q, qy.Boy Bi) = g :: V 5i :: B, 
then f^oM(bl) = l^il- 

(3) \i p = q :: B, with B atomic, and 

Il[s]{\q::B\) = \Q,q::B\ 
then nM{\p\)=m{\Q.<l--B\). 

(4) In all other (trivial) cases, for i = 0, 1, can be arbitrarily defined as or 
True. 

Proposition 4.6.3 (Stability of f^o and r^i). g ||N N|| and Jli g ||N -> S||. 
Proof. Trivial, by proposition 4.6.2. 

□ 

We need first prove that the Iback(T^) plays constructed by 11 are w-correct, when 11 
starts from r2[s]-correct plays of Ta built by $7 (this property correspond to property iv) of 
section 4.6.1). 

Lemma 4.6.1 . Suppose p := Aq :: . . . :: An is a Q[s]-correct play o/Ta- Then 

U[s]{\Ao An\) = \Q,Ao An\ 

and Q,Aq::... :: An is ui-correct. 

Proof. Routine induction on n. If n = 0, then p = Aq. By definition of 11 and ^ 

U[s]i\Ao\) = Msmo\) = \Q,Ao\ 

with Q,Aq cj-correct by construction of ^. 
Suppose now n > 0. By induction hypothesis 

n[s](|^o An-i\) = \Q,Ao :: ... :: A^-i| (4.2) 

and Q,Ao :: ... :: An-i is w-correct. If An-i = 3xB or An-i = B V C, then by definition 
of $7, by equation (4.2) and il[s]-correctness of Aq :: . . . :: An, we have that 

no[s]{\Ao An-i\) = \An\ 

with 

uj{Q,Ao An-i) = Aq . . . An 
By definition 4.6.6 and equation (4.2) 

U[s]{\Ao ::...::An\)=ns]{\Q,Ao A„_i, ^ An\) 

which is cj-correct. If An~i = B AC or An~i = VxB, then 

^s]{\Q,Ao ::...:: An-i,Ao ^nl) 
is automatically w-correct. 

□ 
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We now prove the main theorem of this section: any recursive winning strategy w for 
Eloise in Iback(T^) can be translated into a learning strategy from A for Eloise in Ta_- 

Theorem 4.6.4 (1-Backtracking Strategies into Learning Strategies), is a 
learning strategy for Ta ■ 

Proof. The fact that O satisfies properties 1 and 2 of definition 4.6.3 is trivial and 
follows from proposition 4.6.3. So we prove property 3. Let s be a state and assume 
p = :: ... :: An is a complete 17 [s] -correct play of Ta- Suppose that Aq :: ... :: An ^ W 
and hence An = False. We have to prove that r2i[s]|p| ^ 0. By definition 4.6.7 of il, we 
have 

ni[s]\p\=A[s]{\Q,Ao An\) 

with 

n[s](|^o::...::^n|) = |Q,^o::...:: 
By lemma 4.6.1 \Q,Ao A n\ is tj-correct, since •• • • • •■ is r2[s]-correct. By 

definition 4.6.5 of A, r2i[s]|p| contains all triples 

{E,\Qo,Ao :: ... ::^,|,|Qo,^o - • • • ■.:Ai,Qi,Ao :: . . . :: Ai\) 
not in s such that 

uj{Q,Ao An) = Ao . :: At 

and 

Q,Ao:: ... :: An,Ao Ai = Qo,^o Ai,Qi,Ao Ai 

As implied by very definition 4.6.6 of H, for every j < n, II[s]{\Aq codes a 

play extending the play coded by n[s](|^o Furthermore, for some Q', Q" 

U[s]{\Ao ...■.■.A\) = ^s]\Q',Ao Ai\ = \Q",Ao Ai\ 

So, there is some Q'" such that 

Q",Ao::...::Ai,Q"' = Q,Ao ^„ 

and most importantly, by definition of :: ... :: Ai\ 

XEs\Q",Ao Ai\ = False 

Therefore the triple 

{E,\Q",Ao Ail \Q",Ao A, Q'" ,Ao Ai\) 

belongs to r2i[s]|p|, since it is not in s, by definition of X^;^. 

□ 

4.6.3. Learning Strategies into Realizers. In this section, we prove that the learn- 
ing strategy U for Ta can be translated into a learning based 'PCJ^ciass realizer of A, thus 
proving our main completeness theorem. 

We begin with a bit of coding. 

Definition 4.6.8 . Let : N N and 17^ : N -> Bool be terms of VCFciass such that: 
(1) for every play p :: 3xB of Ta 

nl\p :: 3x^1 = n ^ ^o\p :: 3xB\ = \B{n)\ 
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(2) for every play p :: Bq V Bi 

QqIp :: BqV Bi\= True QqIp :: BqV Bi\ = \Bo\ 

□ 

As a special case of the following definition, we get a candidate realizer for A. 

Definition 4.6.9 (Realizer for A). Let p be an abstract play of Ta- We define by 
induction and by cases a term tp of PCJ-'ciass) with free variables among those occurring 
free in some formula of p, as follows: 

(1) p = q :: Vxi?. 

tq::VxB = tq,-\JxB::B 

(2) p = q :: 3xB. 

tq::3xB = {^o\q ■■ ^xB\,tq:.3xB::B[tl/x]) 

where ti := Jlglg :: 3xB\. 

(3) p = BoVBi. 

tq-.-.BoVBi = {^o\Q ^0 V tq::BoVBi::Bo> ig;;BoVBi::Bi) 

(4) p = BoABi. 

tq::BaABi = (*g;;Bo ABi ::Bo ' *9"Bo ABi ) 

(5) p = q, with q complete. 

tq = Q,i\q\ 

□ 

Lemma 4.6.2 (Completeness Lemma). (1) Let fh = niQ, ... ,mk a sequence of dosed 
stable type-N terms of VCJ-ciass o,nd x = XQ,...,Xk a sequence of variables con- 
taining all the free variables of p :: B. Then 

tp::B[rn/x] G \\[B]\\ 

(2) Let s be a state, m = niQ, . . . ^m^ a sequence of closed type-N terms of VCF ciass, 
X = xq, . . . ,Xk a sequence of variables and {p :: -B)[m[s]/x] a ^l[s]-correct play of 
Ta ■ Then 

tp::B[m/x\ W^s B[rh[s\/x\ 

Proof. We prove (1) by induction on p and by cases. We treat only three representative 
cases, those left out being obvious. 

(1) B = MxC. Let n be a numeral. By inductive hypothesis, we have that 
tp-.-yjxcim/xln = {\x tp;-\jxC::c[m/x\)n 

= tp-,y^xC::c[m,n/x,x\ G ||[C]|| 
Since [VxC] = N — )• [C], we have that 

ip::Vxc[m/x] G ||[VxC7]|| 
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(2) B = 3xC. Let 

ti := nl\p :: 3xC\[m/x\ 

Since the terms in m are stable by hypothesis and JIq is stable by proposition 
4.6.3 and by definition 4.6.8 of figi have that ti is a stable term of type N. By 
inductive hypothesis 

tp:-3xC::C[m,tl/x,x\ G ||[C]|| 

Since \^xC] = N x [C] and 

tp:-3xcWi/x\ = {VlI\p :: 3xC\[m/x],tp.,3^c-.-.c['rn,ti/x,x\) 

we have 

tp,3xc[m/x\ € ||[3xC]|| 

(3) B atomic. Then 

tp::B[rn/x\ = Q.i\p :: B\[rh/x] 

Since [B] = S and Oi is stable by proposition 4.6.3, we have that ip::^!'^/^^] S ll[-S]||- 
We now prove (2) by induction on p and by cases. 

(1) B = VxC. Let n be a numeral. By inductive hypothesis, we have that 

tp::\/xc['rn/x]n = {Xx tp;;\/xC::C[m/x])n 
= tp::\/xC::c['rn,n/x,x] Ih g C['m,n[s]/x,x] 

and hence 

tp::Wxc[lTT- / ^] VxC[?n[s]/x] 

(2) B = 3xC. Suppose 

ti[s] := nl[s]\p :: 3xC\[m[s]/x] = n 
with n numeral. By definition 4.6.8 of Oq, we have that 

r^oHk 3a;C|[m[s]/x] = |C[m[s], n/x, x]| 
and so p :: 3xC :: C[rh[s],n/x, x] is O[s]-correct. By inductive hypothesis 

tp::3xC::c['rn,tl/x,x] llh^ C[m[s], u/ X , x] 

Since 

tp:-3xc[rn/x\ = {^l\p :: 3xC\[m/x],tp.,3xc-.-.c[rn,h/x, x]) 

we have 

tp-Bxcim/x] llhs 3xC['m[s]/x] 

(3) B = Cq A Ci. By inductive hypothesis 

tp-.-.CoACi-.-.Cobri/x] llhs Co[7n[s]/x] 

and 

tp::CoACi::cA^/^] "^s Cl[m[s]/x] 

Since 

tp::Co/\Ci[m/x] = (tp::CoACi::Co["V^]: V:CoACi::Ci[?^/^]) 

we have 

tp::CoACi[m/x] \\\-s Cq A Ci[m[s]/x] 
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(4) 5 = Co V Ci. Suppose 

ti[s] := ^o[s]\p :: Co V Ci\[m[s]/x] = True 
Then, by definition 4.6.8 of 17^, we liave that 

no[s]\q ■■■■ Co VCi|[m[s]/x] = \Co[m[s]/x]\ 
and hence p :: CqV Ci :: Co[w.[s]/x] is f][s] -correct. By inductive hypothesis 

tp-.-.CoVCi-.-.Cobri/x] llhs Co[7n[s]/x] 
Analogously, for ti[s] = False, we have 

tp::CoVCi::Ci[iri'/x] llhs Cl[m[s]/x] 

Since 

ip::CoVCi["V^] = (^^O^b " Co V Cl | [m/x] , tp^^CoVCi ::Co [W^] . *p::Co VCi ::Ci [W^]) 

we have 

ip::CoVCi [m/x] \\\-s Co V Ci[m[s]/x] 

(5) B atomic. Then 

tp::B[in/x][s] = fli[s]\p :: B\[m[s]/x] 

Since p :: B[m[s]/x] is r2[s]-correct and i7 is a learning strategy by lemma 4.6.2, if 
tp--B[m/x][s] = 0; then i?[m[s]/x] = True. 

□ 

Theorem 4.6.5 (Completeness theorem). Suppose there exists a recursive winning 
strategy for player one in Iback(Tyi). Then there exists a term t of VCF ciass such that 
t llh A. 

Proof. By Lemma 4.6.2, point 1 and 2, applied to tA and the empty sequence of 
terms. □ 

4.7. Conclusions 

We have proved a soundness and completeness result for total recursive learning based 
realizability with respect to 1-Backtracking game semantics, solving a conjecture left open 
in Aschieri [4]. 

The contribution of the soundness theorem is semantical, rather than technical, and 
it should be useful to understand the significance and see possible uses of learning based 
realizability. We have shown how learning based realizers may be understood in terms 
of backtracking games and that this interpretation offers a way of eliciting constructive 
information from them. The idea is that playing games represents a way of challenging 
realizers; they react to the challenge by learning from failure and counterexamples. In the 
context of games, it is also possible to appreciate the notion of convergence, i.e. the fact 
that realizers stabilize their behaviour as they increase their knowledge. Indeed, it looks 
like similar ideas are useful to understand other classical realizabilities (see for example, 
Miquel [37]). 

The proof of the completeness theorem has been definitely more technically challenging. 
In our view, moreover, it has two interesting features. In a sense, it is the first application 
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of the ideas of learning based realizability to a concrete non trivial classical proof, which 
is our version of the one given by Berardi et al. [9]. This proof classically shows that if 
Eloise has recursive winning strategy in the 1-Backtracking Tarski game associated to a 
formula A, then she also has a winning strategy in the Tarski game associated to A (but 
a strategy only recursive in an oracle for the Halting problem). Since the existence of this 
latter strategy implies the truth of A, the argument can be seen as a proof of A in some 
version of intuitionistic Arithmetic with EMi. We managed to associate a constructive 
content to this seemingly ineffective proof and found out that it hides a learning mechanism 
to gain correct oracle values from failures and counterexamples. We have then transformed 
this learning mechanism into a learning based realizer of A. Secondly, we have shown 
the interesting theoretical result that backtracking strategies in 1-Backtracking games can 
interpreted as learning realizers. We have thus successfully established a close non trivial 
relationship between two interpretations of classical proofs: game semantics and learning 
based realizability. 



CHAPTER 5 



Constructive Analysis of Learning in Peano Arithmetic 



Abstract. In this chaper we give a constructive analysis of learning as it arises in various 
computational interpretations of classical Peano Arithmetic, such as our learning based 
realizability, Avigad's update procedures and epsilon substitution method. In particular, 
we show how to compute in Godel's system T upper bounds on the length of learning 
processes, which are themselves represented in T through learning based realizability. The 
result is achieved by the introduction of a new non standard model of Godel's T, whose new 
basic objects are pairs of non standard natural numbers (convergent sequences of natural 
numbers) and moduli of convergence, where the latter are objects giving constructive 
information about the former. As foundational corollary, we obtain that learning based 
realizability is a constructive interpretation of Heyting Arithmetic plus excluded middle 
over El formulas (for which it was designed) and of all Peano Arithmetic when combined 
with Godel's double negation translation. As byproduct of our approach, we also obtain 
a new proof of Avigad's theorem for update procedures and thus of termination of epsilon 
substitution method for PA. 

5.1. Introduction 

The aim of tiiis cliapter is to carry out a detailed and complete constructive analy- 
sis of learning, as it arises in learning based realizability for HA + EMi and in Avigad's 
[5] axiomatization of the epsilon substitution method for Peano Arithmetic through the 
concept of update procedure. The importance of this analysis is both practical and foun- 
dational. In the first place, we explicitly show how to compute upper bounds to the length 
of learning processes, thus providing the technology needed to analyze their computational 
complexity. Secondly, we answer positively to the foundational question of whether learning 
based realizability can be seen as an interpretation of classical Arithmetic into intuitionistic 
Arithmetic. 

Our constructive framework is Godel's system T and our metatheory will be purely intu- 
itionistic. Our analysis will be accomplished by restating and then reproving constructively 
the following convergence theorem. 

Theorem 5.1.1 (Convergence). Let t : (N N) N 6e a dosed term of J. Let 
s : N — 7> (N ^ N) be any closed term o/T representing a weakly increasing chain of functions: 
that is, assume that for every numerals n < m, Sn < holds. Then, there exists an n 
such that for all m > n, t{sn) = t{sm)- 

The intuitive meaning of the convergence theorem is the following. It is intended to be 
an analysis of oracle computations. That is, given a non computable function / : N — >• N 

^Define s„ < Sm iff for all immerals I, Sn{l) 7^ implies s„(Z) — Sm{l)- See the premise to definition 
5.2.3 for intuitive meaning. 
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one would like to "compute" t{f). Since this is not effectively possible, in order to obtain 
significant results one may try nevertheless to define a weakly increasing chain s of functions 
with the property that for all numerals n, Sn < /• Such a chain can be seen as a sequence of 
more and more refined approximations of / and can for example be constructed by means of 
learning processes as they arise in learning based realizability or epsilon substitution method 
(see Mints [35]). The theorem says that if t is computed with respect to such a sequence 
of approximations, then a stable answer about the value of t{f) is eventually obtained. 

The convergence theorem is already interesting in itself, but its special significance lies 
in its consequences, which we now describe and shall prove in the final part of the chapter. 
Since most of them cannot be proved if the convergence theorem is not first restated and then 
proven constructively, they provide an important motivation for working in this direction. 

A first consequence of the convergence theorem is that any learning process represented 
by a learning based realizer always terminates. Formally: 

Theorem 5.1.2 (Zero Theorem). Let t be a type S term ofTciass- Define sq := and, 
for every natural number n, s^+i := Sn ^ ^[sn]- Then, there is an n such that i[s„] = 0. 

If the convergence theorem is proven constructively, also the above Zero theorem can 
be and so one obtains a constructive analysis of the numbers of learning steps required to 
complete the learning process. It has as a constructive consequence the following theorem: 

Theorem 5.1.3 (Program Extraction via Learning Based Realizability). Lett 
be a term of Tciass o,nd suppose that t llh \/x^3y^Pxy, with Pxy atomic. Then, from t one 
can define a term u of Godel's system T such that for every numeral n, Pn{un) = True. 

The above theorem sharpens the result obtained in chapter 3 and in Aschieri and Berardi 
[3]. There, we have proved as well that from any t such that t llh \/x^3y^Pxy one can extract 
a computable function v such that for every numeral n, Pn{vn) = True. However, the 
extracted v made use of unbounded iteration, while the u of theorem 5.1.3 is a "bounded" 
algorithm, that is, a program not explicitly using any kind of unbounded iteration. This is 
an important point from a foundational point of view: the algorithms extracted via learning 
based realizability construct witnesses, rather than searching for them. Let us make clear 
that, however, u - from the computational point of view - is equal to v. In fact, u results 
from V just by replacing its only unbounded iteration with a primitive recursive one (of an 
appropriate type). Thus, u just adds to v information about the computational complexity 
of the learning process generated by v. For practical purposes, therefore, v is as efficient as 
u. 

As corollary, one obtains the important result that from classical proofs in Peano Arith- 
metic PA of V3-formulas one can extract bounded algorithms via learning based realizability 
llh. This is done by, first, extracting a realizer from any given proof and, then, by applying 
theorem 5.1.3. In other words, one is able to give a novel proof of the following theorem 
due to Godel (through its Dialectica interpretation, see e.g. [32]): 

Theorem 5.1.4 (Provably Total Functions of PA). //PA \- \/x^3y^Pxy, then there 
exists a term u of Godel's system T such that for every numeral n, Pn{un) = True. 
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The novelty, here, is the technique employed to prove the theorem and the new under- 
standing of extracted programs as realizers able to learn in a constructive way. 

Prom a constructive proof of the convergence theorem one can also provide new con- 
structive proofs of Avigad's [5] fixed point theorem for n-ary update procedures and hence 
of the termination of the epsilon substitution method for PA. Hence, one also obtains a 
constructive analysis of learning in Peano Arithmetic. The novelty, here, is the use of type 
theory to reason about the learning processes generated by update procedures and hence 
epsilon substitution method. 

Theorem 5.1.1 can be proven easily, but ineffectively, in second order logic: 

Proof of theorem 5.1.1 (Ineffective). The informal idea of the proof is the following. 
Terms of system T use only a finite number of values of their function arguments. If we 
"apply" t to the least upper bound fg of the sequence s (w.r.t the relation < of definition 
5.2.3), we find that the finite part of fs effectively used in the computation of t{fs) is already 
contained in some Sfc. So, for every h > k, t{sfi) = t{sk). 

Let us see the details. As proven by Kreisel (for a proof see Schwichtenberg [41]), t has a 
modulus of continuity C, which is a term of system T of type (N — >■ N) — )• N such that the 
following statement is provable in extensional HA'^: 

Vr^", /-^".(Vx" < (C/) fix) = g{x)) ^ tif) = tig) (5.1) 

By using the comprehension axiom, we can define the least upper bound fs of the sequence 
s as follows 

... \ m if 3i such that Siin) = m ^ 
fs\n) = < 

I otherwise 

Let C^^ be the denotation of C in the full set theoretic model M of extensional HA'^ (see 
Kohlenbach [?»2]). Then there exists an n such that for all m > n 

Vx" < (C^'V,) Snix) = S^(X) 

By 5.1, we get that for all m > n, tisn)^^ = i(sm)*^- Hence by soundness of the model 
with respect to formal equality of extensional HA"^, i(sn) and t(sm) normalize to the same 
numeral, since t(sn) = o, and t(sm) = b, with a, b numerals, implies a^^ = t(sn)^^ = 
tism)^ = b^^ and then a = b. 

□ 

The convergence theorem is therefore true, but one cannot hope to prove it construc- 
tively as it is stated. In fact, it is a formula of the form V3V and the simplest incompleteness 
of intuitionistic reasoning as compared to classical reasoning arises precisely for that kind of 
formulas. It is known, for example, that classical finite type Peano Arithmetic PA"^ proves 
the formula y f^~*^3x^yy^ f ix) < fiy), while intuitionistic Heyting Arithmetic HA'^ does 
not. In our case, one could associate to any Turing machine a weakly increasing sequence 
s : N — )• (N — > N) such that for all m, Smin) = if n ^ 0, and Sm(0) = 1 if the machine 
terminates on input n in less than m steps, Sm(0) = otherwise. A constructive proof of 
the convergence theorem relatively to the term A/''~^^/(0) would compute the limit of the 
sequence Xrrt^Smi^), thus determining whether the Turing machine terminates on input n. 
By producing such a sequence s for every Turing machine, we would have a solution for the 
Halting problem. 
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Synopsis of the chapter. In the rest of the chapter, we develop a technology for construc- 
tively reasoning about convergence in Godel's system T and proving a classically equivalent 
form of the convergence theorem. All proofs will be constructive and all their constructive 
content will be made explicit. This constructive effort results in a longer and a bit more 
complex presentation than it could be. However, if one is not interested in full explicit 
details, the techniques used may be simplified in order to yield quite short and powerful 
constructive proofs of the main results of the chapter. 

Our approach has a semantical content. In fact, we starts from considering a kind of 
constructive non standard model for Peano Arithmetic and then we reinterpret Godel's 
system T constants in order to manipulate the new individuals of the model. The reinter- 
pretation of system T will turn out to be particularly suited to perform the computations 
we need to do for constructively reasoning about convergence. From the high level point 
of view, the proof techiques used amount to a combination of Kreisel's no-counterexample 
interpretation and Tait's reducibility /logical-relations method. With the first one, we can 
constructively reason about convergence. With the second, we prove the soundness of the 
model with respect to our purposes. 

In detail, the plan of the chapter is the following. 

In section §5.2, we recall details of Godel system T. 

In section §5.3 we define the first ingredient of our approach, which is a constructive 
notion of convergence for sequences of objects, due to Berardi [6]. It is a no-counterexample 
interpretation of the classical notion of convergence, but it is different from the usual in- 
terpretation. Its main advantage is that it is very efficient from the computational point 
of view, since it enables programming with continuations and hence the writing of powerful 
and elegant realizers of its constructive content, which we will call moduli of convergence. 
Intuitively, a modulus of convergence for a convergent function / : N — )• A will be a term able 
to find suitable intervals in which / is constant; moreover, the length of those intervals will 
depend on a continuation. At the end of the section we use Berardi's notion of convergence 
to reformulate the convergence theorem (see theorem 5.3.1). 

In section §5.4, we introduce the second ingredient of our approach: a model that 
extends the usual full finite type structure generated over natural numbers by replacing 
naturals by pairs {M, f) of a non standard natural number / (which is a function N — t- N 
as in ultrapower models of Peano Arithmetic) and its modulus of convergence N . We also 
syntactically define a semantics (where s is a weakly increasing chain of functions) 
mapping terms of T in to elements of the model and in section §5.5 we show that, thanks 
to we can evaluate every term t : (N — ?• N) — )• N, into a pair {Af,f) such that A/" is a 
modulus of convergence for the function / = Xn^t{sn)- 

In section §5.6, we prove all the corollaries of the convergence theorem that we have 
discussed before. 

5.2. Term Calculus 

In this chapter we will prove results that hold for any "simple" extension of Godel's 
system T (see chapter 2). In this section, we recall the definition and results we shall need 
and introduce some useful notation. 
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Notation. For notational convenience and to define in a more readable way terms of 
type A X B ^ C , for any variables xq : A and xi : B we define 

X{xo,xi)^^^u := Xx'^^^uIttqx/xo ttix/xi] 

where x is a fresh variable not appearing in u. We observe that for any terms tQ,ti 

{X{xo,xi)^''^u){to,ti) = u[to/xo h/xo] 

Often, as in chapter 3, it is useful to add to system T new constants and atomic types, 
together with a set of algebraic reduction rules we call "functional" . 

Definition 5.2.1 (Functional set of rules). Let C be any set of constants, each one 
of some type Ai — > . . . — > A„ — > yl, for some atomic types Ai, . . . , An, A. We say that TZ is 
a functional set of reduction rules for C if 7^ consists, for all c G C and all closed normal 
terms ai : Ai, . . . , a„ : A„ of T, of one and exactly one rule coi . . . a„ i— )• a, where a : A is a, 
closed normal term of T. 

If a system T is obtained from Godel's T by adding a recursive set C of constants and a 
recursive functional set of rules for C, we we call T a simple extension of T. As in chapter 
3, by a standard reducibility argument it can be proved that T is strongly normalizing and 
has Church-Rosser property. Moreover, any atomic-type term of any simple extension T of 
T is equal either to a numeral, if it is of type N, or to a boolean, if it is of type Bool, or 
to a constant of type A, if it is of type A. All results of this paper hold whatever simple 
extension of T is chosen. Let us fix one. 

Definition 5.2.2 (System T). From now on, we denote with T an arbitrarily chosen 
simple extension of Godel's system T. We also assume that T contains constants for deciding 
equality of constants of atomic type. 

Throughout the paper, the intended interpretation of the natural number will be as a 
"default" value. That is, when we do not have any information about what value a function 
has on argument n, we assume that it has value 0. That being said, it is natural to consider 
a function /i : N — )• N to be extending another function /2 : N — )• N, whenever it holds 
that for every n such that /i(n) is a non default value (and hence different from 0), then 
/i(n) = f2{n). f2 may hence have a non default value at some argument where /i has a 
default value, but it agrees with fi at the arguments where /i has not default value. So, /2 
carries more information than /i. 

Definition 5.2.3 (Ordering Between Functions and Terms). Let /i,/2 be func- 
tions N — > N. We define 

/i < /2 ^ V?i G N /i(n) / /i(n) = ^(n) 

Moreover, if ti,t2 are closed terms of T of type N — ?> N representing respectively functions 
5i, 52 : N — >■ N, we will write ti < t2 if and only if gi < g2- 
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In the following, we will write "s S w.i." if is s : N — )■ (N — )■ N) is a closed term representing 
a weakly increasing sequence of functions, that is, if for all numerals n,m, n < m implies 

5.3. The No-Counterexample Interpretation and Berardi's Notion of 

Convergence 

In this paper, we are interested in arithmetical formulas stating convergence of natural 
number sequences. Classically, we consider a sequence of natural numbers to be convergent 
if it is definitely constant, that is, if the there is an element of the sequence which is equal 
to all successive elements of the sequence. Hence, we will consider formulas of the form 

(Vz^) 32;''V/P(z,x,y) (5.2) 

Since that kind of formulas cannot generally be proven constructively, a common standpoint 
is to consider classically equivalent but constructively weak enough statements, as in Kreisel 
no- counterexample interpretation: 

(Vz^) yf^'3x''P{z,x,f{x)) 

If the statement 5.2 (with ^ = N) is provable in PA, then one can constructively extract 
from any proof a term t : (N ^ N) — ?> N of system T such that 

(V/) yf-'^Piz,t{f),f{t{m 

holds (see for example Kohlenbach [32]). In our cases, we have to deal with formulas of the 
form 

3x"V/ > xfix) = f{y) 

where / is a term of type N — t- N, and hence we may be tempted to consider their no- 
counterexample interpretation 

yh^-^V h{x) >x^ fix) = f{h{x)) (5.3) 
If one introduces the notation 

/ i [n, m] = Vx''. n < X < m ^ f{x) = f{n) 
one often finds in literature the following equivalent version of 5.3: 

yh^^V f I [x, h{x)] (5.4) 
which is the no-counterexample interpretation of 

3rE''V/>x/i[x,y] 

While the above notion of convergence 5.4 would be enough for our purposes, it seems 
not to allow straightforward compositional reasoning when one has to deal with non trivial 
interaction of convergent functions. Even when there is no complex interaction, the needed 
reasoning is not direct. For example, one may want to prove that if two functions /, g 
converge in the sense of 5.4, one can systematically find intervals in which they are both 
constant. That is, if 

yh'^^Vf i [x, h{x)] A yh'^^Vg I [x, h{x)] 



5.3. NO-COUNTEREXAMPLE INTERPRETATION AND BERARDI'S NOTION OF CONVERGENCE 77 



then one may want to prove that 

V/i^^^Ba;"/ i [x, h{x)] ^gi[x, h{x)] 

The above imphcation is provable in a non overly complicated way, but when interaction 
increases (as we shall see in proposition 5.3.3 below), one begins to feel the need for a more 
suitable formulation of convergence. 

Berardi [6] introduced a notion of convergence especially suited for managing interaction 
of convergent functions. If one consider the formula 

MzV > ziy"" >xf I [x, y] 

(with the intent of expressing very redundantly the fact that there are infinite points of 
convergence for /) one obtains a very strong notion of constructive convergence by taking 
its no-counterexample interpretation 

VzW^^Bx" > z h{x) > X ^ / i [x, h{x)] 
which after skolemization becomes 

V/i^^^Ba"^" > idVz'' h{z) >z^fi [a{z),h{a{z))] 
which is equivalent to 

V/i"^'' > id3a"^'' > id Vz"/ i [a{z),h{a{z))] (5.5) 
where we have used the notation 

^N^N > id = yx''a{x)>x 

We observe that 5.4 and 5.5 are constructively equivalent. However, from a computational 
point of view, their realizers are quite different: the realizers of 5.5 are able to interact 
directly with each other, as we will see. 

We are now ready to formally define a constructive notion of convergence for sequences 
of numbers: a sequence of objects f : N A is convergent if for any /i""^'* > id there are 
infinitely many intervals [n, h{n)] in which / is constant. 

DEFINITION 5.3.1 (Convergence (Berardi [6])). Let / : N — )• A be a closed term of T, 
with A atomic type. We say that / converges if 

V/i"^" > id Ba"^" > id V// i [a{z),h{a{z))] 

Notation, li t : A ^ B and u : A we shall often write tu in place of tu, for nota- 
tional convenience or for highlighting that is an element of a collection of type-i? terms 
parametrized by terms of type A. 

We now make explicit the constructive information associated to the above notion of 
convergence, through the concept of modulus of convergence. A modulus of convergence 
takes an /i : N — >■ N and returns an enumeration of intervals [n, h{n)] in which / is constant. 
It is a intuitionistic realizer of the notion of convergence. 
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Definition 5.3.2 (Modulus of Convergence). Let / : N ^ be a closed term of T, 
with A atomic type. A term 7W : (N — )• N) — ?> (N — > N) of T is a modulus of convergence for 
/if 

(1) V/i"^" >\dMh> id 

(2) V/i"^" > id VzV I [Mhiz), HMhiz))] 

If /i : N — > N > id and Vz"/ I [Af{z), h{M{z))], M is said to be an h-modulus of convergence 
for /. 

We observe that by definition, if one has a modulus of convergence A4 for a function 
/, one can find an infinite number of intervals of any desired length in which / is constant. 
For example, if one wants to find an interval of length 5, one just defines the function 
h{x) = X + 5 and compute n := Aih{0)- Then, / is constant in [n,n + 5]. Clearly, a 
modulus of convergence carries a lot of constructive information about /. 

5.3.1. Intuitive Significance of the Concept of Modulus of Convergence and 
Restatement of the Convergence Theorem. As we said, Berardi's notion of conver- 
gence works remarkably well when convergent functions interact together, for instance, in 
the definition of a new function. The fact that Berardi's notion is a no-counterexample in- 
terpretation of the classical notion of convergence, explains why it works. We can intuitively 
describe the reasons why it does it well as follows. 

A first reason is purely computational. Given a function hJ^^^ > id and a modulus of 
convergence M, we can interpret the role of h in the computation of A^/i as that of a con- 
tinuation. Constructively, when a new convergent function is defined from other convergent 
functions, one will need to produce intervals in which the new function is constant. One will 
try to achieve the goal by finding intervals in which the functions involved in the definition 
are all constant. The problem is that one may be able to find such intervals for every single 
function, but not for them all together. For example, if one defines the function 

13 := Xx''f{g{x),x) 

then /3 is convergent if g and Xx^ f{n,x) are such for every choice of n. But an interval in 
which g is constant need not be an interval in which /3 too is constant, because we have to 
find some interval in which both g is constantly equal to some m and Xx^f{m, x) is constant. 
We solve the problem through the use of continuations. 

We start by observing that it seems there is a strict sequence of tasks to be performed. 
First, one tries to find an nii such that g is constant in, say, [mi,/i] with mi < li. Then, 
he computes g{mi) = n and pass n to a "continuation" /i : N ^ N which returns an h{n) = 
m2 < h such that Xx^f{n,x) is constant in [m2,^2]- If n^i < "i2 < h, a non trivial 
interval in which g is constant has been found. But if m2 > ^i? Then, g may assume 
different values in all points of the interval [m2, h] and one cannot hope that /3 is going to be 
convergent in [m2, h]- We anticipate the solution contained in the proof of proposition 5.3.3, 
by letting nii = A^fc(O), where is a modulus of convergence for g and, for example, k(x) = 
h{g[x)) + 1. Then, by definition of modulus of convergence, g is constant in [mi,A;(mi)] 
and letting li = k{mi) we obtain that 

m2 = h{n) = h{g{mi)) < h{g{mi)) + 1 = 



5.3. NO-COUNTEREXAMPLE INTERPRETATION AND BERARDI'S NOTION OF CONVERGENCE 79 



as required. In other words, we use k and hence h as continuations, thanks to M. 

The issue we are facing may be further exemplified by the fohowing sequential game 
between k players. Suppose there are convergent functions fi, f2, ■ ■ ■ , fk ot type N ^ N on 
the board and an arbitrarily chosen number m. Players make their moves in order, starting 
from player one and finishing with player k. A play of the game, is an increasing sequence 
of numbers m, mi, m2, . . . , m^, with rrij the move of player i. Player i wins if fi is constant 
in an interval [mk,lk], for some 1^ > mf^. A strategy for player i is just a function h over 
natural numbers, taking the move of the player i — 1 (or the integer m if i = 1) and returning 
the move of player i. The fact that the winning condition depends on the move of player k 
makes very difficult for players 1, ... A; — 1 to win. In this game, each player hopes that in 
the resulting final interval its own function will be constant but his hope is frustrated by 
the following ones, which are trying to accomplish the same task but with respect to their 
own functions. However, player i has a winning strategy effectively computable if he knows 
the strategies of all subsequent players i + I, . . . , k (we cannot assume the trivial winning 
strategy returning the point of stabilization of fi to be effectively computable, since fi is 
arbitrary) 

We are now in a position to tell another reason why moduli of convergence are so useful. 
A winning strategy for player i can be computed by a convergence module. More precisely, it 
can be proved, as consequence of proposition 5.3.2, that if players i + . . . ,k play strategies 
/ij+i, . . . ,hk, then hi := -Mh^o-'-ohi+i is a winning strategy for player i against /ij+i, . . . , /i^, 
whenever is a modulus of convergence for fi. Therefore, if a modulus of convergence for 
each function fi,...,fk is given, one can compute a particularly desirable instance of Nash 
equilibrium, that is, a sequence of functions /ii, /i2, . . . , /ifc such that, if every player i plays 
according to the strategy hi, every play will be won by every player. Therefore, at the end 
of the interaction, every participant will have accomplished its own task. 

We now formulate the promised restatement of theorem 5.1.1 that we shall be able to 
prove. 

Theorem 5.3.1 (Weak Convergence). Let t : (N — )• N) — ;> S 6e a closed term of T, 
with S atomic type. Then we can effectively define a closed term : (N ^ (N — )• N)) — > 
(N — ;> N) (N N) of T, such that the following holds: for a// s : N (N N), s G w.i. 
and numerals n, M.s is a modulus of convergence for Xm^tn{sm)- 

5.3.2. Basic Operations with Moduli of Convergence. We now prove a couple 
of propositions, both to illustrate the use of moduli of convergence and to provide lemmas 
we will need in the following. First, we show that given two terms /i and /2, if each 
one of them has a modulus of convergence, then there is a modulus of convergence that 
works simultaneously for both of them. In particular, we can define a binary operation 
U between moduli of convergence such that, for every pair of moduli A4,M, U is 
"more general" than both M and M. Here, for every Mi,A42, we call A^2 more general 
than A4i, if for every term /, if A4i is a modulus of convergence for / then also 7W2 is a 
modulus of convergence for /. We this terminology, we may see Al UAA as an upper bound 
of the set {A4,M}, with respect to the partial order induced by the relation "to be more 
general than". The construction of the pair MhoNh^-^h below may also be seen as a Nash 
equilibrium for the two player version of the game we have discussed above. 
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Proposition 5.3.2 (Joint Convergence). Let M and M be moduli of convergence re- 
spectively for fi and f2 ■ Define 

Then MuAf is a modulus of convergence for both fi and f2- 
Proof. Set 

C:=MUM 

First, we check property 1 of definition 5.3.2. For all li}^^^ > id, Nh > id by definition 5.3.2 
point (1) and so h o J\fh > id. Thus, for all /i"^" > id and 

since M. has property (1) of definition 5.3.2 and hence M-hoMk — Therefore, for all 
/i""^" > id, > id and we are done. 

Secondly, we check property 2 of definition 5.3.2. Fix a term h^~^^ > id and a numeral z. 
We have that 

fi i [Mho^fJz),hoMh{MhoX,Sz))] (5.6) 
since is a module of convergence for fi. Moreover, 

/2 i Wh{MhoMA^)),h{J^h{MhoM,S^m (5.7) 

since 7\A is a modulus of convergence for /2. But the starting point of the interval in 5.7 is 
greater or equal to the starting point of the interval in 5.6, for Afh > id, while their ending 
points are equal. Hence also 

/i i [MhiMhoMA^)),hiMh{MhoMd^m 

and hence both fi and /2 are constant in the interval [Ch{z),h{Ch{z))] by definition of C. 

□ 

We now consider a situation in which a family {/n}nGN of convergent terms interacts 
with a convergent term g and we show the result of the interaction is still a convergent 
term. In the following, we call "object of type A" any closed normal term of type A. 

Proposition 5.3.3 (Merging of Functions). Let f : A ^ ^ A) be a closed term, 
with A atomic, and 7\A : ^ — ?• (N ^> N) — )■ (N — ?• N) be such that for every object a of type A, 
Ma is a modulus of convergence for fa- Let moreover g : N ^ A and let M be a modulus of 
convergence for g. Define 

ni{M,M,g) := A^''-''AzX(-^ho^^(^)) 

with 

Then 7ii{M,Af,g) is a modulus of convergence for 

Ara" (n) 
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Proof. Property (1) of definition 5.3.2 follows by the same reasoning used in proposi- 
tion 5.3.2. We check property (2) of definition 5.3.2. Set L := 'Hi{Jv[ , J\f , g) . The idea is 
that C has to produce an interval i in which g is constant and equal to a, while the interval 
produced by Ma in which fa is constant will be contained m. i. C does the job by using 7V^ 
as a continuation. 

Fix a term closed /i^^" > id and z a numeral. We have that 

9 i[MhoMl^{z),hoM'y,{MhoMl^{z))] (5.8) 
since is a module of convergence for g. In particular, 

g i[M'j,{Mh,M;S^))MK{MhoMi{m (5-9) 
Say that for all n in the intervals in 5.8 and 5.9, g{n) = a. By definition of 

[K{MhoM^^{z))MK{MhoM'Sm 

= [Nah{MhoMl^{z)),h{^ah{MhoMl^{z)))] (5.10) 

Since Ma is a modulus of convergence for fa, we have 

fa i [Mah{MhoM'S^)),h{Mah{MhoM'S'^)))] 

But for all x in the interval 5.10, 

(An"/g(.„)(n))x = fa{x) 

Hence 

\n^fgin){n) i [Mi{MhoMi{^))MK{MhoM'Sm 
and so f g[n){''^) is constant in the interval \Ch{z),h{Ch{z))] by definition of C. 

□ 

5.4. Computations with non Standard Natural Numbers 

For technical convenience we add now to system T a constant (t> : N — >■ N with no 
associated reduction rules. In this way, each term t : A can be viewed as functionally 
depending on 0, but it is still considered as having type A, instead the more complicated 
(N ^ N) ^ A. Of course, terms of atomic type are not in general equal to a constant or a 
numeral, if they contain 0. 

Definition 5.4.1 (Evaluation at u). Let i be a term. For any term li : N N, we 
denote with t[u\ the term t[u/'^]. 

Adopting this notation, what we want prove is that \i t : A, with A atomic, and s G 
w.i., then the function \m^t[sm\ constructively converges, that is, it has a modulus of 
convergence. A natural attempt for achieving the goal is to recursively decompose the 
problem. For example, suppose we want to study the convergence of the function 

*{+tit2) := Am" + tit2[sm] : N ^ N 

where s G w.i.. and + : N ^ N — ?> N represents a constant of T encoding the operation 
of addition of natural numbers. Since ti : N and ^2 : N may have complex structure, it is 
natural to recursively study the functions 



since A/"^ > id. 
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*ti := Xm^ti[sm] : N N 

and 

*t2 := Xm^t2[sm] :N^N 

But if we want to study the function *(+tit2) as a combination of *ti and *t2, it is clear 
that + cannot be interpreted as itself, but as a function *+ of *ti and *t2- We would like 
the following equation to hold 

*i+ht2) = *+*tlt2 

As a consequence of our notation, also the following equation must be true for all numerals 
n 

*{+tit2)in) = +Ct,in))i*t2in)) 

These considerations impose us to define 

*+ := A^r^Asf^^Am" + <7i(n)<72(n) 

At a first look, this may seem a rather strange way of doing computations. But it turns out 
that it is strongly not the case. *ti and *t2 may be interpreted as hypernatural numbers 
and *+ as the operation of addition of hypernaturals as they are defined in ultrapower non 
standard models of Peano Arithmetic. 

5.4.1. Non Standard Models of Arithmetic. The first non standard model of 
Arithmetic is due to Skolem [42]. The universe of that model is indeed made of functions 
N — 7> N, but we instead describe a variant of the Skolem construction, which is the ultrapower 
construction (see for example Goldblatt [23]). 

Fix a non principal ultrafilter over N. First, define an equivalence relation ~ between 
functions N — ?> N as follows: 

fi^f2 ^ {x e N I = f2{x)} G T 

(The intuition here is that an ultrafilter collects the "big" subsets of N and hence two 
functions are to be considered equal if they have equal values for "great many" arguments. 
For example, two functions which, as sequences, converge to the same natural number are 
considered equal, for they agree on a cofinite set of N, which must belong to every non 
principal ultrafilter). Secondly, define 

*N := (N ^ N)^ 

that is, *N is the set of all natural number functions partitioned under the equivalence 
relation ~. Finally, set 



*0 


:= Xn% 




*s 


:= Xn^S{n) 




*+ 


:=A/i"A/2^^V/i(n) 


+ /2(n 




:=A/f^f^A/2^^V/i(n) 


• /2(n) 



where S,+,- are the usual operations over natural numbers. In general, if one wants to 
define the non standard version of a standard function / : N*^ — )> N, he simply lets 



/ := A/f-^ . . . A/r''An^/(/i(n), . . . , A(n)) 
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It can be proved that the structure 



is a model of Peano Arithmetic as similar to the usual structure of natural numbers as to 
satisfy precisely the same sentences which are true under the usual interpretation. Formally, 
it is elementarily equivalent to the structure of natural numbers. 

Elements of *N are usually called hypernatural numbers. Since they are so similar to 
natural numbers, it perfectly makes sense to think about defining a model of system T 
over hypernaturals. Indeed, Berardi [7] used hypernaturals, under a weaker equivalence 
relation, to construct an intuitionistic model for Aq maps and Berardi and de' Liguoro [12] 
used them to interpret a fragment of classical primitive recursive Arithmetic. 



5.4.2. A non Standard Model for the System Tq. In order to approach gradually 
our final construction, we first give a definition of a non standard model for Tq, which is 
Godel's T restricted to having only a recursion operator R of type N— )-(N— >-N^N)— 7>N^N 
and choice operator if : Bool ^ N — ?> N ^ N. Hence, Tq represents the primitive recursive 
functions. 

The definition of the model is purely syntactical and this is the key for our approach to 
go through. In fact, we are defining an internal model, that is a representation of Tq into 
T itself. First, define the new type structure as: 



*N := N N 
*Bool := N -> Bool 

*{A B) := *A *B 
*{A X B) := *A X *B 



From a semantical point of view, we interpret natural numbers as functions. Since the 
construction is syntactical, there is no need to describe an equivalence relation between those 
functions. But, accordingly to which equivalence relation one has in mind, the definition we 
are going to give will make sense or not from the semantical point of view. For the results 
of this chaper, we have no utility in putting extra effort to define a model for Tq, which 
is also a model for Peano Arithmetic. Hence, we may assume that *N represents just all 
functions over N without any partition. 

Now, for every term u : T, define a term *u of type *T by induction as follows 
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*0 
*True 
*False 
*S 
*if 
*R 
*{x^) 
*iut) 
*{Xx^u) 
*{u,t) 

*{TTiU) 

with the type B of Rb equal to *N ^ (N ^ *N *N) ^ N ^ *N. 

The definition of the constants and the functions *S and *if is exactly the one used in 
the construction of ultrapower models of natural numbers. The definition of *R is different 
because involves higher type arguments, but it is a straightforward generalization of the 
ultrapower construction. Intuitively, *R/i/2(7 has to iterate /2 a number of times given by 
g. But since g is now an hypernatural number, the concept "5 times" makes no direct 
sense. Hence, *R also picks as input a number m, transform g into g{m) and iterates /2 a 
number of times given by g{m). But since /2 is of type *N ^ *N — ?• *N, the function given 
to Rb is not directly /2, but a term An'*/2(Ax"n) that transforms n into its hypernatural 
counterpart and gives it as the first argument of f2- After all this work is done, one obtains 
a hypernatural 

h := Rij/i(An''/2(Ax"n))9(m) 

So if *R stopped here, it would not return the right type of object. Hence, it returns h{m), 
consistently to the fact that g has been instantiated to m previously. 

The above construction can be generalized to Godel's T, with a little more effort to 
be put in the generalization of *R and *if to all types. A version of T just manipulating 
hypernaturals is not enough for our purposes, and will be included in our final construction, 
so details are postponed to the next sections. 

5.4.3. System T over Hypernaturals with Moduli of Convergence. In the con- 
text of this work, we are not interested into the whole collection of hypernatural numbers, 
but only in those who are convergent. Moreover, we want also to produce, for each one of 
these convergent hypernaturals, a modulus of convergence. The idea therefore is to put more 
constructive information into the model of hypernatural numbers and to define operations 
that preserve this information. The new objects we are going to consider are hypernatural 
numbers with moduli of convergence. They can be represented as pairs 

(AA,/) 

where / : N — t- N is an hypernatural number and : (N — t- N) ^> (N ^ N) is modulus of 
convergence for /, as in definition 5.3.2. The resulting model will be a full type structure 
generated over this basic objects and their equivalent in the other atomic types. We will 



= Am True 

= Am" False 

= A/*''Am" S(/(m)) 

= A5*^°°^A/;"A/2"Am" if/(m)/i(m)/2(m) 

= A/;"A/?^*''^*''A/''Am" (RB/i(AnV2(Ax"n))g(m))(m) 

= x^ 

= *u*t 

= Xx'^*u 

= {*u,*t) 

= ■Ki*U 
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call it the model of hypernaturals with moduli. In the following, for any s Gw.i., |n]s will 
be the denotation of a term u of T in this new model and the aim of this sections is to 
syntactically define the interpretation function {-Js- 

In order to construct such a model, we will have to define new operations that, first, 
generalize the ones over hypernaturals we have previously studied and, secondly, are also 
able to combine moduli of convergence. 

For example, how to define the non standard version of addition? The summands 
are two objects of the form (A/i, /i) and (^2, /2)- The second component of the sum will 
be the non standard sum 

/i*+/2:=AmVi(m) + /2(m) 

of /i and f2- The first component will be a modulus of convergence for /i*+/2, and so a 
simultaneous modulus of convergence for both /i and /2 is enough. From proposition 5.3.2, 
we know how to compute it with U from A/i and 7V2. We can thus define 

[+l,(A/-i,/i)(A/'2,/2) := (A/'iUA/-2,/i*+/2) 

We now launch into the definition of our syntactically described model for the whole 
system T. First we define the intended interpretation M^^ of every type T. 

Definition 5.4.2 (Interpretation of Types). For every type T of system T, we define 
a type My by induction on T as follows. 

(1) T = A, with A atomic. Then 

Ma := ((N ^ N) ^ (N ^ N)) x (N ^ A) 

(2) T = A^B. Then 

Ma^b ■■= Ma Mb 

(3) T = AxB. Then 

Maxb ■■= Ma X Mb 

If A is atomic, the interpretation of A is the set of pairs formed by a function N ^ ^ 
and its modulus of convergence (if it happens to have one). This is an accord with our view 
that whenever a s Sw.i. is fixed, a term t of atomic type can be interpreted as a function 
Xm^t[sm] paired with a modulus of convergence. The model of hypernaturals with moduli 
can be seen as the collection of sets denoted by types Mt, for T varying on all types of T. 

We now define a logical relation between the terms of our intended model of hyper- 
naturals with moduli and the terms of system T. It formally states what properties any 
denotation of any term of T should have. It formalizes of our previous description of what 
the model should contain. 

Definition 5.4.3 (Generalized Modulus of Convergence). Let t and M be closed 
terms of T and s Gw.i.. We define the relation A4 gmCg t - representing the notion "A4 is 
a generalized modulus of convergence for f - by induction on the type T of t as follows: 

(1) T = A, with A atomic. Let M : M^. Then 
A4 gmCg t A4 = {jO,g), £ is a modulus of convergence for g and g '= Xn^t[sn] 
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where we have defined {g = An^t[s„,]) = for all numerals m, g{m) = t[sm]- 

(2) T = A-^ B. Let M : Ma^b- Then 

M gmCg t (Vn^. M gmc^ u =^ MM gmc^ tu) 

(3) T = AxB. Let M : MaxB- Then 

M gmCg t ^=^> {ttoM gmCg vrot A ttiM gmc^ vrit) 

The aim of the rest of this section is to syntactically define a semantic interpretation 
of the terms of T into the model of hypernaturals with moduli, such that for every 
term u : A and s Gw.i., lujs ginc^ u. This means that, if A is atomic, u is evaluated in a 

pair {C,g) such that g =* \n^u[sn] and £ is a modulus of convergence of g. Then, if given 
any term t : (N — )• N) ^ A, we set u := t<t> and consider [ujs, we automatically obtain a 
constructive proof of theorem 5.3.1. 

In the following, we will make repeated use of the fact that the notion of generalized 
modulus of convergence is consistent with respect to equality. 

Lemma 5.4.1 (Equality Soundness). Suppose Mi gmc^ h, Mi = M2 and ti = t2. 
Then M2 gmc^ i2- 

Proof. Trivial induction on the type of T. 

□ 

We now define a fundamental operation on moduli of convergence. The construction is 
a generalization of the one in proposition 5.3.3. 

Definition 5.4.4 (Collection of Moduli Turned into a Single Modulus). Let 
M : A Mt and {M,g) : Ma, with A atomic. We define by induction on T and by cases 
a term n{{M,g),M) of type My. 

(1) T atomic. Then 

n{{M,g),J\f) := {ni{M,Xa^7ro{Ma),g), An%(„)(n)) 
with / := Xa^TTiMa and 7ii as in proposition 5.3.3. 

(2) T = C ^ B. Then 

n{{M,g),M) := \C''^n{{M,g),Xa^MaC) 

(3) T = C X B. Then 

n{{M,g),Ar) := {n{{M,g), Xa'^7roMa),n{{M,g), Xa^TTiMa)) 

If we call "object of type A" any closed normal term of type A, then the role of the 
term Ti is to satisfy the following lemma, which is one the most important pieces of our 
construction. It provides a way of constructing the semantics of a term ut, with t of atomic 
type A, if one is able to define a semantics for t and for ua for every object a of type A. 
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Lemma 5.4.2 . Let u and t he closed terms respectively of types A ^ T and A, with A 
atomic. Suppose that for every object a of type A, Ma gmc^ ua and {A4,g) gmc^ t. Then 
U{{M,g)M) gmc, ut. 

Proof. By induction on T and by cases. 

(1) T atomic. We have 

n{{M,g),M) := {'Hi{M,\a^M^fa),g),Xn^fgin)in)) 

with 

/ := \a\iMa 

and 

g = Xn^t[sn] 

for by hypothesis {M.,g) gmc^ t. Moreover, for every object a of type A 

fa =* \n^ua[sn] 
since by hypothesis Ma gmc^ ua. We must show that 

-HiiM.Xa^TiQMa^g) 

is a modulus of convergence for the function )^ri^ f g{n){n) and that \n^fg{^n) 
\n}^ut[S'n\. For this last part, indeed, for every numeral m, there is an object 
a = g{m) such that 

{^n^ fg{n){n))m = fa{m) 

=* u[sm]{a) 
= u[sm]{g{m)) 

=* u[sm]{{\n^t[sn\)m) 

= u[Sm]{t[Sm]) 

= {\ri^ut[sn])m 

Now, since {Ml,g) gmc^ t, Ml is a modulus of convergence for g. Moreover, for 
every object a of type A, Ma gmc^ ua by hypothesis, and therefore iroMa is a 

modulus of convergence for An"* na[s.„] =* fa- By proposition 5.3.3, we obtain that 

ni{M,Xa^7roMa,g) 
is modulus of convergence for f g(n){n) and we are done. 

(2) T = C ^ B . Let v : C and suppose C gmc^ v. We have to show that 

n{{M,g),M)C = n{{M,g),Xa^MaC) gmc, utv 
But for every object a of type A, Ma gmc^ ua. Therefore, for every object a of 
type A 

MaC gmCg uav = {\m'^umv)a 
By induction hypothesis 

TidMjg), Xa'^MaC) gmc^ {Xm^umv)t = utv 
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which is the thesis. 



(3) T = C X B. We have to show that, for i = 0, 1, 



7rM{M,g),M)=TTi{n{{M,g),Xa^ ^oAA, ),n{{M, g), Xa^ i^Ma ) ) 
= n{{M,g),Xa^TTiJ\fa)) gmc, Tri{ut) 

Now, for every object a of type A, Ma gmc^ ua. Therefore, for every object a of 
type A 

T^i-Xfa gmc^ T^i{ua) = {Xm^TTi{um))a 
By induction hypothesis 

^.{{M, g) , Xa'^TTiAfa) gmc^ (Am^7rj(nm))t = 7rj(ut) 
which is the thesis. 

□ 

We are now in a position to define for each constant c of T a term |c]s, which is 
intended to satisfy the relation [c]s gmc^ c. |c]s can be seen as the non standard version of 
the operation denoted by c. 



Definition 5.4.5 (Generalized Moduli of Convergence for Constants). We de- 
fine for every constant c : T a closed term [cj^ : M^, accordingly to the form of c. 

(1) c : A, A atomic. For any closed term u of atomic type, define 
Then 



fcjs := M;d,c 

(2) c = cD : N ^ N. Let 

M := Xr/ {Xh^^^ Xm^\f Sm{n) = Sfi{jfi){n) then m else h{m), Xm^ Sm{n)) 
Then 

ms :=X{M,g)''''n{M,g),M) 

(3) c 7^ 0, c / if , c : ^0 ^ • • • ~^ A, with A, Ai atomic for i = 0, . . . , m. 
If m > 0, then define 



{4s := A(£o,5o>''-^« • • • A(£„,5„^)''^™ (£0 U £1 U . . . U C^, 

Xn^c{go{n)) . . . 

assuming left association for U. 
If m = {c : Aq ^ A), define 



icjs ■.= X{M,gr-o{M,Xn''cig{n))) 
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(4) c = Rt, Rt recursor constant with T = A^{'N^A^A)^'N^A. Define 

J\f := An" RuI{Xn''CMid,n)n 

with 

[/ := Ma ^ (N ^ Myi ^ Ma) ^ N ^ Ma 

Then 

(5) c = ify with T : Bool ^ A ^ A ^ ^. Define 

:= A6^°°^if 5 then Ci else £2, 

Then 

lifrl. := X{M,g)''^-^X/:'f^XC^^n{{M,g),f/) 

The definition of [cj^ is a generalization of the operations done with hypernaturals. 
In case (1), we transform basic objects into their hypernatural, hyperboolean and hyper- 
constant counterparts (we call them hyperobjects) all paired with their trivial moduli of 
convergence. 

In case (2), the interpretation [0]^ of <t> is obtained by first defining uniformly on the 
numeral parameter n a collection of interpretations Mn = I'^njs of <i>n, and then using the 
term 71 to put together the interpretations in such a way that [0]s(A^,5) interprets [4>t]s 
whenever {M,g) = 

In case (3), we provide the non standard version of the function represented by c, which 
is a function |c]s which combines both hyperobjects and their moduli of convergence. 

In case (4) and (5) we have generalized the ideas of subsection 5.4.2. In particular, for 
T = A^{}l^A^A)^N^A and A atomic, the definition of [RtIs is exactly the 
same of *R in subsection 5.4.2, enriched with the information of how to combine moduli of 
convergence. In fact, if we consider the term 

lRTjsIC{M,g) 

= {ni{M, An"^o (ATn) , g) , AnV^Cn) (n) ) 

with / := An"7ri(7V„), its right projection is equal to 

Xn^fg{n){n) 

which is equal to 

An"(7riRc/X(An"/:Xid,„)5(n))(n) 
which correspond exactly to the term 

*R/i/25 = An" (RB/i(A?i''/2(Ax"n))g(n))(n) 

of subsection 5.4.2. 

We now prove that for any constant c, [cj^ is a generalized modulus of convergence for 

c. 

Proposition 5.4.1 . For every constant c, {cjs gmc^ c. 

Proof. We proceed by cases, accordingly to the form of c. 
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1) c = 0. Let t : N and suppose {Ai,g) gmc^ t. We have to prove that 

Ms{M,g) gmc, cDt 

By definition 5.4.5 of [ct)]^ 

ms{M,g) = 71^,9), Af) 

with 

J\f := Xn^ {Xh^^^ Xm^\i Sm{n) = s^„^^{n) then m else h{m), Xm^ Sm{n)) 

Since {AA,g) gmCg t, if we prove that for every numeral n, Afn gmc^ 0n, we obtain 
by lemma 5.4.2 that 'H{{M,g),M) gmc^ $t and we are done. So let us show that, 
given a numeral n, Ti^Nn is a modulus of convergence for the function 

vriTVn = Am''sm(n) =* Am" 4>(n)[sm] 
We have to prove that given any closed term h^^^ > id and numeral n, 

Xm^Sm{n) i [(7roA/'„)h(2;), /i((7roA/'„)h(^;))] 
We have two possibilities: 

i) Sz{n) = Since s G w.i., we have either = and so 

V/. z<y< h{z) =^ Sy{n) = 
or Sz{n) = Sh(z)i^) 7^ and so 

Vy". z <y < h{z) =^ Sy{n) = Szin) 

Therefore 

Xm^Sm{n) i[z,h{z)] 

= [iT^oMn)h{z), h{{7ro{Mn)h{z))] 

by definition of Af. 

ii) Sz{n) 7^ Sh(z)i^)- Since s S w.i., we have Sz < s/i(^) and hence = Sz{n). So 

= Sh(h(z)){n) and as above 

= [ ( vro AA„ ) ( z ) , /i ( ( vr A/; ) ft ( z ) ) ] 

2) c / 0, c / if, c : Ao ^ • • • ^ ^ ^. 

i) m > 0. Suppose : and {Ci,gi) gmc^ tj for all i = 0, . . . , m. We have to 
prove that 

|c]s(£o,5o) • • • {Cm.gm) gmCg Cti . . . 

We have that gi An''tj[s„] for i = 0, . . . ,m. Moreover, since by definition 5.4.5 

of Ms 

= (£0 U £1 U . . . U An''c(goH) . . . {gm{n))) 
we must show that 

7ro(|c]s(£o,ffo) • • • {^m,gm)) 

=£0 u /:i u . . . u 
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is a modulus of convergence for 

\n^c{go{n)) . . . {gm{n)) 

"= c{ti[Sn]) . . ■ itm[Sn]) 
= An"cti . . . tm[Sn] 

Since for i = 0, ... ,m, Ci is a modulus of convergence for gi, by repeated applica- 
tion of proposition 5.3.2 we deduce that CqUCiU. . is modulus of convergence 
for all gi, . . . ,gm simultaneously. Hence for all closed terms /i : N — > N > id and 
numerals z, and for i = 0, . . . , ?n 

gi i [{Co U /:i U . . . U Cm)hiz), h{{Co U /:i U . . . U Cm)h{z))] 
and therefore 

Am" c(5i(m)) . . . {gn{m)) i [{Cq U £1 U . . . U £.^)hiz),h{{Co U £1 U . . . U Cm)hiz))] 
which is the thesis. 

ii) m = 0. Straightforward simplification of the argument for i). 

(3) c : A, A atomic. By definition 5.4.5 

We have therefore to prove that Xh^~^^ Xm^m is a modulus of convergence for Xn^c, 
which is trivially true, and that An"c[sn] = Xn^c, which is also trivial. We conclude 
|c], gmc, c. 

(4) c = Rt, Rt recursor constant with T = A— )-(N— t-^— ^^d)— >N— Suppose 
1 gmCg u : A, C gmc^ v : H ^ A ^ A and {M,g) gmc^ t : N. We have to prove 
that 

lRThIC{M,g) = ni{M,g),M) gmc, Rruvt 

where 

Af := An" Rt/Z(An"i:7Wid,„)n 

If we show that for all numerals n, Nn gmCj, Rtuvu, by lemma 5.4.2 we obtain 
that 

n{{M,g),M) gmc, Rruvt 

We prove that by induction on n. 
If n = 0, then 

Mo = RuI{Xn^CM\d,n)0 = I gmc^ u = RtuvO 
If n = S(m), then 

Afs(m) = RuAXn''CM;d,n)S{m) 

= (An"/:Xid,„)m(Rc/X(An"/:>Jid,n)m) 
= £7Wid,„^(Rc/X(An"/:>^id,„)m) 

= CMid,mMm 
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By induction hypothesis, Mm gmc^ RTUvm. Moreover, A1id,m gmc^ m and by hy- 
pothesis C gmCj, V. Hence 

C"M.\d,m-N'm gmc^ vm{RTUvm) = TZTUvS{m) 
which is the thesis. 

(5) c = ifr, with T : Bool A ^ A ^ A. Suppose Ci gmCg ui : A, C2 gmCg U2 : A 
and {M,g) gmc^ t : Bool. We have to prove that 

l\fT}s{M,g)CiC2 = n{{M,g),M) gmc, iMnms 

where 

M ■= A6^°°iif 6 then Ci else £2 

If we show that for all a € {True, False}, Afa gmc^ {Xb^°°-^\fTbuiU2)a, by lemma 
5.4.2 we obtain that 

l-L{{M,g),N) gmCg (A6^ifr6nin2)t = \^TtuiU2 

We prove that by cases. 
If a = True, then 

Ma = C-i gmCg ui = {\h^\^TbuiU2)a 

If o = False, then 

Ma = C-2 gmCj, U2 = {\b^\^TbuiU2)a 
Hence, we have the thesis. 

□ 

We are finally ready to define the interpretation of every term of T in our model of 
hypernaturals with moduli. 



Definition 5.4.6 (Generalized Moduli of Convergence for Terms of T). For 
every term f : T of system T and s €w.i., we define a term \v\s '■ My by induction on v and 
by cases as follows: 

(1) f = c, with c constant. We define |c]s as in definition 5.4.5. 

(2) V = x^^ X variable. Then 

{x% := x^-^ 



(3) V = ut. Then 

(4) V = Xx'^u. Then 

(5) V = {u, t). Then 

(6) V = TTiU. Then 



lutjs := {uUtls 
lXx\js := Ax^^ M, 

iTTiUjs := TTilujs 
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5.5. Adequacy Theorem 

We are now able to prove our main theorem. For every closed term u, lujs is an 
inhabitant of the model of hypernaturals with moduli of convergence. 

Theorem 5.5.1 (Adequacy Theorem). Let w : A be a term of T and let x^^ , . . . ,x^" 
contain all the free variables of w. Then, for all s ^w.i. 

AxJ^^i . . . Xxn"""^ lw\s gmc, \x^^ . . . Xx^"w 

Proof. Let ti : Ai, . . . ,tn ■ An be arbitrary terms. We have to prove that 
Ml gmc, ti,...,Mn gmc, t„ =^ lw}s[Mi/x^^^ ...Mn/xn^"] gmc, w[ti/xf\ . .tn/x^"] 
For any term v, we set 

V :=v[ti/xf^ ■■■tn/x^"] 

and 

MI := {vUMi/xJ^ . . . 

With that notation, we have to prove that Iwjs gmc^ w. The proof is by induction on w 
and proceeds by cases, accordingly to the form of w. 

(1) w = c, with c constant. Since {cjs is closed and c does not have free variables, by 
proposition 5.4.1 

Ms = I4s = {cjs gmc^ c = c = w 

which is the thesis. 

(2) w = xf\ for some 1 <i <n. Then 

PI7 = x"^^^ [A^i/x^^^ . . . Mn/xn-'-] = Mi gmc, ti = xf^[t,/xf' ■ ■ ■ = w 

which is the thesis. 



(3) w = ut. By induction hypothesis, [u]s gmCg u and {tjs gmc^ t. So 

futjs = MsMs gmc^ ut = w 

which is the thesis. 

(4) w = Xx^u. Let t : A and suppose M gmCg t. We have to prove that [luj^ gmc^ wt. 
By induction hypothesis 

{Xx^ulsM = {Xx^^l4s)M = \4s[M/x^^\ gmc, u[t/x^] = wt 
which is the thesis. 



(5) w = {uq,ui). By induction hypothesis, |uo]s gmc^ uq and {uijs gmc^, ui. There- 
fore, for i = 0,1 

TTil{uo,Ul)js = TTiduojsluijs) = {Uijs gmCg Ui = TTjltJ 

which is the thesis. 
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(6) w = TTjU, with i G {0, 1}. By induction hypothesis, [nj^ gmc^ u. Therefore, 

{iTiUls = TTilujs gmC^ TTiU = W 

which is the thesis. 

□ 

5.6. Consequences of the Adequacy Theorem 

In this section, we spell out most interesting consequences of adequacy theorem. 

5.6.1. Weak Convergence Theorem. We can finally prove the constructive version 
of theorem 5.1.1, our main goal. The following theorem is even stronger of the previously 
enunciated theorem 5.3.1, because it states that one can find moduli of convergence for any 
uniformly defined collection of terms. 

Theorem 5.6.1 (Weak Convergence Theorem for Collection of Terms). Let 
t : N ^ (N — 7- N) — 7- S be a closed term of T not containing 0, with S atomic type. Then 
we can effectively define a closed term M : N (N (N -)• N)) (N N) (N N) 
ofT, such that for all s : N ^ (N ^ N), s € w.i. and numerals n, A4nS is a modulus of 
convergence for Xm^tn{sm)- 

Proof. Let 

M := A/As''^("^'')vro((Ax^»It,«01,)A^id,,) 
By the adequacy theorem 5.5.1, 

A2;^«[t^NCt)], gmc, Ax"t^.»0 

Since for every numeral n, A^id,n gmc^ n, we have 

(Ax""It,N01,)Xid,„ gmc, t„0 

By definition of generalized modulus of convergence and of ^A, MnS is a modulus of con- 
vergence for Xm^tn'^[Sm] = Xm^tn{Sm)- 

□ 

5.6.2. Learning Based Realizability and Provably Total Functions of PA. If a 

is a state of knowledge, we mantain the notation 

u[a] := [ippa/^tip XP^^f^P addpcr/Addp] 

of chapter 3, since there is no confusion with the notation u[v] of the previous section, which 
was defined for v of type N — >■ N. 

Theorem 5.6.2 (Zero Theorem for Collections of Terms of Tclass)- Let t : N — ;> S 
be a closed term ofTciass, where S is the atomic type of knowledge states. Then, there exists 
a term Zero : N — > S ofTheam such that for every numeral n, t„[Zero„] = 0. 

Proof. We may assume t contains as oracles only constants Xp,<t>p,Addp for some 
fixed predicate P. The general case is analogous and involves only a little bit more of trivial 
coding. 

In the first place, we have to carry out some simple coding. We have to represent states 
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of knowledge by terms of type N — >■ N. This is straightforward since a state represents a 
function over N. Define 

/ := Xa^Xn^\f XP^f^ then Lppan + 1 else 

/ takes a state a and returns the function coding it; if n is a numeral, when xp^''^' = True, 
faU returns ifpan + 1 and not just ippan, in order to ensure that is returned only when 
it is just a trivial value, i.e. when xp<7 = False. 

Given a term (7 : N — )■ N, intended to represent a state a, define terms ipp, Xp, addp, intended 
to code respectively 93p(T, xp<7, addpcr, as follows 

(/3p := An'*if g{n) = then else g{n) — 1 
Xp '■= An'*if g{n) = then False else True 
addp := An'^Am'^if g{n) = then addp07T,m else 
Moreover, for every term n, define 

u3 := u[ipl,/<Pp xf>/Xp add^Addp] 
It is easy to see that for all terms u and all states a, u[a] = u^" . 

We can now give the important part of the argument. Fix a numeral n. Let '■= and 
fjm+i := Cm ^ ^nicm] be a recursive definition of a sequence of states (which can be coded 
in T). Our goal is to write down a term of system Thc&m which is able to find a state (Jk+i 
such that tri[ffc] = in[ffc+i]: as we will see, this condition implies t[(Tfc_|_i] = 0. Let 

U := Am^A/^^C 

By the weak convergence theorem 5.6.1 applied to U, there exists a term M of TLoam such 
that, for every numeral I and s G w.i., Mis is a modulus of convergence for Xm^Ui{sm)- 
Set s := (Xm^fa^). Then s G w.i., since ctq < ai < a2 ■ ■ ■ ■ Therefore A4nS is a modulus of 
convergence for Xm^ lAri{sm)- If we choose h := Am''m + 1 and set k := Mnsh, we have that 
Un{sk) = Un{sk+i) by definition of modulus of convergence. We thus obtain, by definition 

of U ad s, that t'n'' = tn '"^^ and so tn[<7fc] = tn[cfc+i]. Let 

Zero := Xm^ a (^Mmsh)+i 

We have 

Zero„ lyj t„[Zero„] =fjfc+i y t^k^+i] 

= {ak^tn[(Jk])mtn[ak] 

= 

= Zero„ 

Since t„[Zero„] is consistent and disjoint with Zero„, we conclude t„[Zero„] = and obtain 
the thesis. 

□ 

As anticipated in the introduction, from learning based realizers we can extract algo- 
rithms of system T. 
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Theorem 5.6.1 (Program Extraction via Learning Based Realizability) . Let 
t be a term of Tciass cLnd suppose that t llh Vx^By^Pxy, with P atomic. Then, from t 
one can effectively define a term u of Gddel's system T such that for every numeral n, 
Pn{un) = True. 



Proof. Let 

V := \m^iTi{tm) 

V is of type N — > S. By theorem 5.6.2, there exists a term Zero : N — )■ S of TLcam such that 
Vn\Z-&rOri\ = for every numeral n. Define 

w := Am''(7ro(tm)[ZerOm]) 

and fix a numeral n. By unfolding the definition of realizability with respect to the state 
Zero„, we have that 



and hence 



that is to say 



and therefore 



tn llhzero„ ^y^Pny 



vri(tn) lll-zero„ Pn{wn) 



7;„[Zero„] = Pn[wn) = True 



Pn{wn) = True 

We observe that : N ^ N and w is a term of TLcam- By standard coding of states into 
natural numbers and of all other constants of TLcam into terms of Godel's T, one can code 
every term of TLeam into Godel's T. Hence, there exists a term u of Godel's T such that for 
all numerals n, u{n) = w{n), which is the thesis. 

□ 



Remark 5.6.3 . We point out that the term u of theorem 5.6.1, modulo some trivial coding 
of states into numbers, bears a strong resemblance to the term t from which it is defined. In 
particular, u is straightforwardly obtained from a modulus of convergence Ai that carries 
the constructive information associated to the convergence of t. In turn, A4 is obtained via 
the translation which just replaces type-^ constants and variables of t with new terms 
of type M^. As an instance, recursion constants are replaced with recursion constants 
RM4. Therefore, the type of recursion goes through a constant increase of 2, since Ma is 
obtained by changing the basic types C with Mc*. 

We conjecture it is not possible to amend our translation as to preserve the types of recursion 
constants. This should be due to Avigad's theorem: if one is able to find zeros of finite 
update procedures, than one can compute every provably total function of PA. But in the 
next subsections, we show how to compute finite zeros of update procedures in Godel's 
system T, thanks to moduli of convergence. If the translation did not increase the 
recursion type, then the term computing the zero of an update procedure would have the 
same recursion level of the latter. But since primitive recursive functionals are enough as 
update procedures, one would get a contradiction to Avigad's theorem, because one could 
interpret all provably total functions of Arithmetic with primitive recursion. 
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We are now able to prove a version of the classic theorem of Godel, characterizing the 
class of functions provably total in PA as the class of functions representable in system T. 

Theorem 5.6.4 (Provably Total Functions of PA). //PA ^\/x^3-ifPxy, then there 
exists a term u of Godel's system T such that for every numeral n, Pn[un) = True. 

Proof. Starting from the assumption that 

PA h Mx^^y^Pxy 

by Kolmogorov double negation translation (see for instance [43]), we have that 

HA h Vx"^^3/Pxy 

Therefore 

HA + EMi h Vx''3/Pxy 
and so there is a term t of Tciass such that 

t llh Vx"3/Pxy 

By theorem 5.6.1, there exists a term u of Godel's system T such that for all numerals n 

Pnu{n) = True 

□ 

5.6.3. Zeros for Update Procedures. Thanks to the adeguacy theorem, we are able 
to give a new constructive proof of Avigad's theorem for update procedures. Here, we give 
a slightly different definition of update procedure. This is not a limitation, since the update 
procedures which are actually used by Avigad [5] in proving 1-consistency of PA still fall 
under the following definition. 

Definition 5.6.1 (Update Operator, Typed Update Procedure). Fix a primitive 
recursive bijective coding |_| : (N^ U {0}) ^ N of and of triples of natural numbers into 
natural numbers. Define a binary operation © which combine functions / : N — t- N and codes 
of the form |(l,?i, m)| of pairs of natural numbers and returns a function N — t- N as follows 

/ © \{l,m, n)\ := Ax^if x = m then n else f{x) 

For convenience, define also / © \0\ = f. 

A typed update procedure of ordinal k £ N (also said a k-ary typed update procedure) 
is a term Z// : (N — > N)'^ — t- N of Godel's T such that the following holds: 

(1) for all sequences f = fi, ■ ■ ■ , fk of closed type-N — )• N terms of T, Uf = \ n, m)| =^ 
l<i<k. 

(2) for all sequences f = fi, . . . , fk and g = gi, . . . , gk of closed type-N — >• N terms of T 
and for all 1 < i < k, if 

i) for all j <i, fj = gj; 

n) Uf = \{i,n,m)\, gi{n) = m and Kg = \{i,h,l)\ 
then h n. 
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If W is a fc-ary typed update procedure, a zero for W is a sequence / = /i, . . . , of closed 
type-N N terms of T such that Uf = 

Every unary update procedure gives rise to a learning process, i.e. a weakly increasing 
chain of functions. 

Proposition 5.6.5 (Learning Processes from Unary Update Procedures). Let 
U he a unary update procedure and define by recursion sq := o"^" := Ax^'O and Sk+i ■= 
Sk^BUsk- Then s Gw.i.. 

Proof. Suppose Sj(n) = m ^ 0. We have to prove that for all j, Si+j{n) = m. We 
proceed by induction on j. Suppose j > 0. Since sq = O'*'^'' and Si{n) ^ 0, it must be that 
for some < i, Usi^ = |(l,n,m)|. By induction hypothesis, Si+j_i(n) = m. By definition 
5.6.1, point 2-ii), Usi^j^i / for all I. Since 

it must be that Si-^j{n) = m. 

□ 

We first prove that unary typed update procedures have zeros. 

Theorem 5.6.6 (Zero Theorem for Unary Typed Update Procedures). LetU : 
(N —7- N)'^"''^ — > N 6e a term of T such that for all closed type-]^ — t- N terms fi, ■ ■ ■ , fk of T, 
Z^/i ■ ■ ■ /fc 'is a typed unary update procedure. Then one can constructively define a closed 
term £ : (N — )■ N)'^ — >■ (N — >■ N) ofT such that for all closed type-N — >■ N terms fi, ■ ■ ■ , fk ofJ 

Ufi---fk{efi...fk) = |0| 

Proof. First, for any term /i : N — > N, define 

£'^=A(A^,<7)^"(X,An''%(n))) 

The same proof of proposition 5.4.1 (in the case of constants of type N ^ N) shows that for 
all closed terms /i of T and s €w.i., gmc^ h. Define 

Ms := Xhl^"" . . . xhl^^'iuisd''^ ...c}'^ Ms 

and fix closed type-N — )• N terms fi, - ■ ■ ,fk of T. By the adeguacy theorem 5.5.1, for all 
s Gw.i., lUjs gmc^ U and hence 

Ms := Msfi ...fk gmc, Ufi... fk<^ 

So, for all s G w.i., 7ro(A4s) is a modulus of convergence for Xm^Ufi . . . fk{sm)- Define 
by recursion a term s such that sq := O""^" and Sn+i := Sn © ^/i . . . fkSn- Then s Gw.i. 
by proposition 5.6.5, since Ufi...fk is a typed unary update procedure. So -kqIM-s) is 
a modulus of convergence for Am" Ufi . . . fk{sm)- If we choose h := Xm^m + 1 and set 
j := TTo{A4s)h, we have that 

^/i • • • fk{sj) = Ufi . . . fk{sj+i) 
by definition of modulus of convergence. So let 
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e/l ■ ■ ■ fk '■— •S(7ro(A4s)/i) + l 

Then 

•Si+2 = Sj+i ®Ufi . . . fkisj+i) 

= {sj ®Uh... fk{sj)) ®Ufi... fk{sj+i) 
= {sj ®Ufi... fk{sj)) ®Ufi... fk{sj) 
= Sj®Ufi...fk{sj) 

= Sj+i 

and hence it must be that 

Kfi... fkiefi ...h)=Uh... fk{sj+i) = |0| 

which is the thesis. 

□ 

We are now able to prove the zero theorem for n-ary typed update procedures, following 
the idea of Avigad's original construction. 

Theorem 5.6.7 (Zero Theorem for n-ary Typed Update Procedures). Let U : 
(N — )■ N)*^ — )> N 6e a typed update procedure of ordinal A; € N. Then one can constructively 
define terms ei, . . . ,ek of Gddel's T such that 

Usi . . .efc = |0| 

Proof. By induction on k. The case k = 1 has been treated in theorem 5.6.6. There- 
fore, suppose k > 1. Define 

:= Xg^i-^"" ...Xgl-^f\fUgi...gk^i = \ik,n,m)\ then \{l,n,m)\ else |0| 

Since for all closed type-N — > N terms /i, . . . , of T, U^fi ■ ■ ■ fk-i is a unary typed update 
procedure, by theorem 5.6.6, we can constructively define a term of T such that for all 
closed type-N N terms /i, . . . , f^-i of T 

Z^fc/i • • • • • • /fc-i) = |0| 

and hence 

Ufi... fk-i{ekfi--- fk-i) / \ {k,n,m)\ 

This implies that 

\gl-^''...\gl^fUgi...gu^^{egi...gk-i) 

is a typed update procedure of ordinal k — 1. By induction hypothesis, we can constructively 
define terms ei, . . . ,£k~i of Godel's T such that 

Uei... £k-i{£kei ■ ■ ■ £k-i) = |0| 

which is the thesis. 

□ 

An important corollary of theorem 5.6.7, is the termination of the epsilon substitution 
method for first order Peano Arithmetic. 
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Theorem 5.6.8 (Termination of Epsilon Substitution Method for PA). The H- 

process (as defined in Mints [35]) of the epsilon substitution method for PA always termi- 
nates. 



Proof. See Avigad [5] 



□ 



CHAPTER 6 



Learning in Predicative Analysis 



Abstract. We give an axiomatization of the concept of learning, as it implicitly appears 
in various computational interpretations of Predicative classical second order Arithmetic. 
We achieve our result by extending Avigad's notion of update procedure to the transfinite 
case. 



6.1. Introduction 

The aim of tiiis chapter is to provide an abstract description of learning as it is appears 
in various computational interpretations of predicative fragments of classical second order 
Arithmetic. Our account has a twofold motivation and interest. 

Its first purpose is to provide a foundation that will serve to extend learning based real- 
izability to predicative fragments of Analysis: a possible path to follow is the one suggested 
here. In particular, we describe the learning processes that arise when extending the ap- 
proach of learning based realizability to predicative Arithmetic and prove their termination. 
This is achieved by introducing the notion of transfinite update procedure. 

Secondly, we continue the work of Avigad on update procedures [5] and - as anticipated 
- extend them to the transfinite case. The concept of transfinite update procedure may be 
seen as an axiomatization of learning as implicitly used in the epsilon substitution method 
formulated in the work of Mints et al ([35], [36]). The notion is useful to understand 
the core of the epsilon method and its fundamental ideas without having to deal with the 
complicated formalism and non relevant details. Moreover, as interesting byproduct of 
the conceptual analysis of the epsilon method through transfinite update procedures, one 
can provide a combinatorial statement that is equivalent to the 1-consistency of various 
fragments of predicative Arithmetic in a very scalable way. In particular, the informal 
statement "U(a): every transfinite update procedure of ordinal less than a has a finite 
zero" very rapidly acquires logical complexity even at small ordinals. For example: 

(1) U(2) corresponds to the 1-consistency of HA + EMi. 

(2) U(n + 1), with n € w, corresponds to the 1-consistency of HA + EM„ (excluded 
middle over formulas). 

(3) U(u;) corresponds to the 1-consistency of PA 

(4) U(w • 2) corresponds to the 1-consistency of EA (Elementary Analysis) 

(5) U(a;'^) corresponds to the 1-consistency of PA^ plus A]^-comprehension axiom. 

(1), (4) are treated in this thesis, (3) has been proved by Avigad [5], (2) will follow by 
extending learning based realizability. (5) should follow by straightforward extension of the 
methods we will use to prove (4). In order to make precise the statement U(a) and prove 
refined versions of (l)-(5), one has to choose a formal system in to which represent update 
procedures. All the update procedures used in this thesis may be assumed to be represented 
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in system T. 

Plan of the Chapter. In section §6.2 we introduce and motivate the concept of transfinite 
update procedure and give a very short non constructive proof of the existence of finite zeros. 

In section §6.3 we explain the notion of "learning process generated by an update pro- 
cedure" and prove that every learning process terminates with a zero for the associated 
update procedure. The result represents a more constructive proof of the existence of finite 
zeros and the learning processes are "optimal", in the sense that one could provide con- 
structively the expected ordinal bounds to their length and of the size of finite zeros (by 
applying techniques of Mints [35]) 

In sections §6.4 and §6.5 we formalize the notion of update procedure in typed lambda 
calculus plus bar recursion and prove the existence of zeros for update procedures of ordinal 
less than by writing down simple bar recursive terms. 

In section §6.6 we devote ourselves to a case study: we show that U(a; • 2) implies the 
1-consistency of EA by proving that it implies the termination of //-processes of the epsilon 
substitution method for EA. 

6.2. Transfinite Update Procedures for Predicative Analysis 

From the computational point of view, classical predicative second order Arithmetic 
poses very difficult problems. Axioms of comprehension ask for functions g able to decide 
truth of formulas: 

a/^W. g{x) = O 

Axioms of (countable) choice ask for functions g computing existential witnesses of truth 
of formulas: 

(Vx^B/ <Pix,y)) ^ 3/-^VxXx,ff(x)) 

A Kleene-style realizability interpretation for even the most simple form of the excluded 
middle 

EMi : Vn".V/^Pny V 3x^Pnx 

asks for deciding the truth of semidecidable formulas of the form 3x^ Pnx, with P decidable. 

In general, learning based computational interpretations of predicative fragments of 
classical analysis (such as our learning based realizability, epsilon substitution method, up- 
date procedures, Herbrand analysis) provide answers to the above computational challenges 
by the following three-stage pattern: 

(1) They identify a sequence F - possibly transfinite - of non computable functions 

(2) They define classical witnesses for provable formulas by using programs recursive 
in F. 

(3) They define learning procedures through which it is possible to find, for every 
particular computation, a suitable finite approximation of the functions of F such 
that one can effectively compute the witnesses defined at stage two. 

The functions in the F of stage (1) are the computational engine of the interpretation. 
Given the difficulty of computing witnesses in classical Arithmetic, they are always non 
computable. It is therefore not surprising that given this additional computational power, 
one is able to define at stage (2) witnesses for classical formulas. If we picture the sequence 
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-F as a sequence of infinite stacks of numbers, the learning process of point (3) finds a 
"vertical" approximation of F: functions of F are infinite stacks of numbers whereas their 
finite approximations are finite stacks. Moreover, a crucial point is that the sequence F 
is not an arbitrary sequence. In a sense, F is also "horizontally" approximated: for every 
ordinal a, the recursion theoretic Turing degree of Fa is approximated by the degrees of 
Fp, for /3 < a. This property is very important: in this way, the values of the functions in 
F can be gradually approximated and learned. 

More precisely, F can be seen a sequence of functions obtained by transfinite iteration 
of recursion theoretic jump operator (see for example Odifreddi [38]). That is, for every /3, 
if /3 is a successor, Fjj has the same Turing degree of an oracle for the halting problem for 
the class of functions recursive in Fp-i (jump); if /? is limit, Fp has the same Turing degre 
of the function mapping the code of a pair (a,n), with a < /3, into Fa{n) (join or /3-jump). 
A fundamental property of such a sequence is that the assertion Fi3(n) = m depends only 
on the values of the functions F^, for a < /3, and the values of Fp are learnable in the limit^ 
by a program g recursive in the join of F^ for q < /3, which is a guarantee that learning 
processes will terminate. 

We now give an informal example of the kind of analysis which is needed to carry out 
the first stage of a learning based interpretation, in the case of EA. A complete treatment 
is postponed to section 6.6. 

Example 6.2.1 (Elementary Analysis). Consider a subsystem of second order Peano 
Arithmetic in which second order quantification is intended to range over arithmetical sets 
and hence over arithmetical formulas (formulas with only numerical quantifiers and possibly 
set parameters). Since one has to interpret excluded middle over arbitrary formulas, it 
is necessary to provide at least programs that can decide truth of formulas. Numerical 
quantifiers correspond to Turing jumps. That is, if we have a program t (with the same 
function parameters of (j)) such that for every pair of naturals n, m 

t{n,m) = True <;=^ (j){n,m) 

then the truth of 

3x'*(/>(n, x) 

is equivalent to the termination of a program Q{n) exhaustively checking 

t(n,0),t(n,l),t(n,2),... 

until it finds - if there exists - an m such that t{n, m) = True. Applying the jump operator to 
the Turing degree t belongs to, one can write down a program xt which is able to determine 
whether Q{n) terminates. That is 

Xt(?^) = True <;=^ 3x''(/)(n, x) 

Similarly, one eliminates universal numerical quantifiers, thanks to the fact that V = 
Iterating these reasoning and applying 2k times the jump operator - and given a recursive 
enumeration (f)Q,(f)i, . . . ^ of arithmetical formulas - one can obtain for every Sg;. formula 

(t)n{m) := Bx^Vy" • • ■ ^x^Vy" P(m, xi, yi, . . . , Xk.yk) 
a program t„ such that 

tn{m) = True <;=^ 4>n{m) 
^In the sense of Gold [22]; Fp{n) = m <;=^ limfe_>oo gin, k) = m 
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Using the cj-jump operator, one can write down a program u such that 

u{n,m) = True <J=^ tn{m) = True 

and hence 

u(n,m) = True <J=> (j)n{m) 

Now a S} formula 

- provided we assume that J^^^ooi j-g^j^ggg over arithmetical predicates - can also be ex- 
pressed as 

3n\[Xm''u{n,m)/f] 

So applying again a jump operator to the recursive degree of ti[Xm^ u{n, m) / f], one is able 
to write a program determining the truth value of 3f^^^°°-^(j)i. Iterating this reasoning, one 
can decide the truth of arbitrary formulas. 

Summing up, in order to decide truth in Elementary Analysis, one needs to apply the jump 
operator u + uj times and thus produces a sequence F of non computable functions F of 
length uj + uj. All the programs that we have described are recursive in some initial segment 
of F. 

We are now in a position to understand the following axiomatization of the learning 
procedures cited in point (3) above. 

Definition 6.2.1 (Transfinite Update Procedures). Let a > 1 be a numerable 
ordinal. An update procedure of ordinal q is a function Z^/ : (a — (N ^ N)) ^ (a x N x 
N) U {0} such that: 

(1) U is continuous, i.e. for any f : a (N ^ N) there is a finite subset ^ of a x N 
such that for every (7 : a — ?• (N — )■ N) if f-y{n) = ^^(n) for every (7,^) € A, then 
Uf = Ug. 

(2) For all functions /, 5 : a ^ (N ^ N) and every ordinal /9 € a, if 

i) for every ^ < /S, = g^- 

ii) Uf = (/3,n,m), gp{n) = m and Ug = 
then i ^ n. 

The concept of transfinite update procedure is a generalization of Avigad's notion of 
update procedure [5]. A transfinite update procedure, instead of taking just a finite num- 
ber of function arguments, may get as input an arbitrary transfinite sequence of functions, 
which are intended to approximate a target sequence F; as output, it may return an update 
{I3,n,m), which means that the /3-th function taken as argument is an inadequate approx- 
imation of Fi3 and must be updated as to output m on input n. Condition (2) in definition 
6.2.1 is a little bit stronger than Avigad's requirement, which would be: 

(2)' For all functions /, (7 : q — ?■ (N — ?• N) and every ordinal /3 G a, if 
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i) for every 7 < ^, = g^- 

ii) Uf = {/3,n,m), gpn = m, fp < gp and Ug = (where fp < gp is defined as 
fp{x) 7^ implies fp{x) = gp{x)); 

then i ^ n. 

But in fact, the update procedures which are actually used by Avigad [5] in proving 1- 
consistency of PA still fall under our definition. 

Condition (2) means that the values for the /3-th function depend only on the values of 
functions of ordinal less than /? in the input sequence and an update procedure returns only 
updates which are relatively verified and hence need not to be changed. In this sense, if 
Uf = (/3,n,m), one has learned that Fp{n) = m; so if gp is a candidate approximation of 
Fp and gpin) = m, then Ug does not represent a request to modify the value of gp at point 
n, whenever / and g agree on all ordinals less than /?. 

We remark that the choice of the type for an update procedure is somewhat arbitrary: we 
could have chosen it to be 

{a^ {X ^Y)) ^ {ax X xY)U {0} 

as long as the elements of the sets X and Y can be coded by finite objects. Since such coding 
may always be performed by using natural numbers, we choose to consider X = y = N. 

The use of transfinite update procedures made by learning based computational inter- 
pretations of classical arithmetic can be described as follows. Suppose those interpretations 
are given a provable formula with an attainable computational meaning, for example one of 
the form Vx^By" Pxy, with P decidable. Then, for every numeral n, they manage to define 
a term t„ : (a — )■ (N — t- N)) ^ N and an update procedure Un of ordinal a such that 

Unif) = =^ Pn{tn{f)) 

for all / : a — >■ (N — 7- N). The idea is that a witness for the formula 3y^Pny is calculated 
by tn with respect to a particular approximation / of the sequence F we have previously 
described. If the formula Pn{tn{f)) is true, there is nothing to be done. If it is false, then 
i^nif) = i(^,n',m) for some f3,n,m: a new value for Fp is learned. This is what we call 
"learning by counterexamples" : from every failure a new positive fact is acquired. We have 
studied an instance of this kind of learning in the chapter on learning based realizability 
for HA + EMi (for the case of a = 1), when we defined realizability for atomic formulas: in 
that case the pair (n, m) was produced by the realizer of the excluded middle. We will see 
another instance in section 6.6 in the case a = to + k, with /c G N: the triple (/3, n, m) will 
be produced through the evaluation of axioms for epsilon terms. 

The effectiveness of the above approach depends on the fact that every update procedure 
has a finite zero, as defined below. 

Definition 6.2.2 (Finite Functions, Finite Zeros, Truncation and Concatena- 
tion OF Function Sequences). Let U be an update procedure of ordinal a. 

(1) / : a — 7- (N N) is said to be a finite function if the set of (7, n) such that /^n 7^ 
is finite. 
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(2) A finite zero for U is a finite function / : a ^ (N ^ N) such that Uf = 0. 

(3) Let / : a (N N) and /3 < a. Let /</3 : /3 (N N) be the truncation of / 
at /3: 

:= 7 G /3 ^ A 

(4) Let ai, a2 be two ordinals, /i : ai ^ (N ^ N) and /2 : a2 (N N). Then the 
concatenation /i * /2 : (ai + 02) — )■ (N ^ N) of / and g is defined as: 



f^{n) if 7 < ai 

5^(n) if 7 = ai + /3 < ai + 02 



(5) With a slight abuse of notation, a function / : N — )• N will be sometimes identified 
with the corresponding length-one sequence of functions 1-^ (n G N 1-^ /("■))• 



We now prove that every update procedure has a finite zero. We will give other more 
and more constructive proofs of this theorem, that will allow to compute finite zeros for 
update procedures and thus witnesses for classically provable formulas, thanks to learning 
based interpretations. But for now we are only interested into understanding the reason of 
the theorem's truth and give a very short non constructive proof. All the subsequent proofs 
can be seen as more and more sophisticated and refined constructivizations of the following 
argument. 



Theorem 6.2.3 (Zero Theorem for Update Procedures of Ordinal a). LetU he 
an update procedure of ordinal a. Then lA has a finite zero. 



Proof. We define, by transfinite induction, a function / : a ^ (N — )• N) as follows. 
Suppose we have defined /-y : N ^ N, for every 7 < /3. Define the sequence /</? : j3 — >■ 
(N ^ N) of them aU 

:= 7 G /3 ^ A 

Then define 



' if \Jg{<^-P)^{^^^) U{f<p *g)^{P,x,z) 

y otherwise, for some y such that 3gi°'~l^)^i^~^^) U{f<^p * g) = x, y) 



By axiom of choice and classical logic, for every /3, and fp are well defined. So we can 
let 

Suppose U{f) = {I3,x,z), for some (3 < a: we show that it is impossible. For some 
h:{a-il3 + l))^{N^ N), f = f^p^fp* h. Hence, for some 5 : (a - ^) ^ (N ^ N) 

U{f<l3*9) = {I3,x,y) A /^(x) = y 
by definition of /. But U is an update procedure and so 

{i^{f<l3* g) = {I3,x,y) A fp{x) = y AU{f<fi* fl3*h) = {/3,x,z)) ^ x^x 

which is impossible. We conclude that hl{f) = and, by continuity, that U has a finite zero. 

□ 
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6.3. Learning Processes Generated by Transfinite Update Procedures 

In this section we show that every update procedure lA generates a learning process and 
this learning process always terminates with a finite zero of lA. This result is an abstract 
version of the termination of the ff-process as defined in the various versions of epsilon sub- 
stitution method (see Mints et al. [35]). The proof of termination is non-constructive and 
is similar to the one in Mints et al. [35] (which however is by contradiction while ours is not). 

If lA is an update procedure and U{f) = (7, n,m), then the value of at argument n 
must be updated as to be equal to m. But as explained in the introduction, we may imagine 
that all the values of all the functions fp, with /3 > 7, depend on the values of the current 
f^. Therefore, if we change some of the values of we must erase all the values of all the 
functions for /3 > 7, because they may be inconsistent with the new values of fjs. In a 
sense, / is a fragile structure, that may be likened to an house of cards: if we change some 
layer all the higher ones collapse. We define an update operator © that performs those 
operations. 



Definition 6.3.1 (Controlled Update of Functions). Let / : a ^ (N — )• N) and 
(7, n, m) G a X N X N. We define a function / © (7, n, m) : q (N — > N) such that 



(/ © {'y,n,m))p{x) 



ffs{x) if /3 < 7 or (/3 = 7 and x 7^ 
m if 7 = /3 and x = n 
otherwise 



n 



We also define f®%:=f. 



We now define the concept of "learning process generated by an update procedure Z^/" . 
It may be thought as a process of updating and learning new values of functions, which 
is guided by lA. It corresponds to the step three of the learning based computational 
interpretations of classical arithmetic we have described in the introduction. Intuitively, 
such a learning process starts from the always zero function 0" . \ilA says that some value 
of 0" must be updated - i.e. U{Q°') = (7, n,m) - then the learning process generates 
the function lA^^^ := 0" © (7, n,m). Similarly, if lA says that some value of must be 
updated - i.e. U{U^^^) = (7' ,n',m') - then the learning process generates the function 
ZY(2) := ^/(l) © (7' ,n',m'). The process goes on indefinitely in the same fashion. 



Definition 6.3.2 (Learning Processes Generated by U). Let lA be an update pro- 
cedure of ordinal a. For every n € N, we define a function Z/^^"^ : a ^ (N ^ N) by induction 
as follows: 

^(0) — 0" := 7 e a ^ (n G N ^ 0) 

Moreover, a function / : a ^ (N — )■ N) is said to be lA-generated if there exists an n such 
that f =U^"'\ 
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The aim of the learning process generated by lA is to find a finite zero for lA. Indeed, if 
for some n, Z^(ZYW) = 0, then for all we thus say that the learning 

process terminates. We now devote ourselves to the proof that learning processes always 
terminate. In other words, every update procedure U has a Z//-generated finite zero. 

Given an update procedure U, its useful to define a new "simpler" update procedure, 
obtained from U by fixing some initial segment of its input, ignoring all updates relative to 
this fixed part of the input and adjusting their indexes. 



Definition 6.3.3 . Let U be an update procedure of ordinal a. Then, for any function 
g : (3 ^ {N ^ 'N), with /3 < a, define a function 

Z^g : ((a - /3) ^ (N ^ N)) ^ (a - /3) X N X N U {0} 

as follow 

11 (f\ - j (7, if * /) = {P + l,n,m) 
I w otherwise 

(We point out that ii f3 = = Ug = U as it should be) 



Indeed Ug as defined above is an update procedure. 



Fact 1 . Let U be an update procedure of ordinal a. Then, for any function g : (3 
(N N), with p <a: 

(1) Ug is an update procedure of ordinal a — /3. 

(2) For every : N ^ N, Ug,h = {Ug)h. 



Proof. Immediate. 

□ 

The strategy of our termination proof can be described as follows. Given an update 
procedure U of ordinal a, we shall define a sequence of functions g : a ^ (N — ?• N) such 
that a "reduction lemma" can be proved: if, for some /3 < a, Ug^^ has a W^^^-generated 
finite zero, then for some 7 < /3 also Ug^^ has a iYg^^ -generated finite zero (for definition of 
5<^, recall definition 6.2.2 ). But the greater the ordinal j3 the easier is to compute with 
a learning process a finite zero for ^^^^ because the sequence g^p becomes so long that 
the input for lA^p becomes short. So we shall be able to show that for some large enough 
(3, 13 < a, Ug^p has a W^^^-generated finite zero, which proves the theorem in combination 
with the reduction lemma. This technique can be seen as a generalization of Avigad's [5] 
to the transfinite case. 

We now prove the reduction lemma in the limit case. 



Lemma 6.3.4 (Reduction Lemma, Limit Case). LetU be an update procedure of ordinal 
a and g : (3 ^ {H ^ 'H) , with (3 limit ordinal and f3 < a. Then 

(1) /// : {a — (3) — 7- (N — 7- N) is Ug-generated, then there exists j < (3 such that 0^~'^ * f 
is Ug^ -generated. 
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(2) IfUg has a Ug- generated finite zero, then there exists 7 < /? such that Ug^^ has a 
^9<i -generated finite zero. 

Proof. (1) Let n be the smallest among the i such that / = Ug\ If A; < ?i and 
UgiU^''^) = 0, then 

So we have that for all k <n, UgiUg'^^) 7^ 0. Since /3 is a limit ordinal and U is continuous, 
there exists a 7 < /3 such that for every k < n 

and 

U{g<^ * O'^-T * f) = {5,n,m) A6 < p =^ S <j (6.1) 
We prove by induction on k < n that 

y<7 9 

which is the thesis. If A; = 0, 

y<7 9 

If /c + 1 < n, then for some 5, 1, m 

Ug{U^^^) = {5,l,m) 

Then, by definition oiUg, 

U{g*U^g''^) = {/3 + 5,l,m) 

and hence 

Z^(5<7 * O'^-^ * iYf )) = {f3 + 5, 1, m) = (7 + (/3 - 7) + 6, /, m) 
Therefore, by definition of Ug^^ 

Ug,^{0^-^*U^g''y) = {{P-j) + 6,l,m) 
By the help of induction hypothesis, we conclude that 

= Ul%®Ug,,{Q^-^*U^'^) 

= Ul%®{{P-l) + 5,l,m) 

= (0''-^*Z^f))e((/3-7) + 5,/,m) 

= 0'^-^*(Z^f)e((5,/,m)) 

(2) We continue the previous proof. Suppose that / is also a finite zero of Ug. If 

= * /) = U{g<^ * 0^-' * f) 

then by definition of Ug^^ 
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Therefore, suppose 

{6,l,m)=U{g*f)=U{g^^*0('~^*f) 
Since Ug{f) = 0, by definition of Ug we have that 6 < (3. By 6.1, we have also that (5 < 7. 
So, by definition of Ug^^ 

which is the thesis. 

□ 

We now prove the reduction lemma in the successor case. 

Lemma 6.3.5 (Reduction Lemma, Successor Case). LetU be an update procedure of 
ordinal a. Define (7 : N — > N as follows: 



g{x) :-- 

Then: 



'y if 3i. UiU^'^) = (0, x,y) M = m\n{n \ 3z UiU^""^) = (0, x, z)} 
otherwise 



(1) For every finite function qq < ^g, if go*0°' ^ is U- generated and f isUgg-generated, 
then go * f is U -generated. 

(2) IfUg has a Ug- generated finite zero, thenU has a U-generated finite zero. 

Proof. (1) By induction on the number m such that Ug^^ = f. If / = Ug^^ = 0"^^, 
then go * f = go * 0"~^ is ^/-generated by hypothesis. 
If 

then go *Ugl^ is ^-generated by inductive hypothesis, i.e. for some n, go 
have two cases: 

i) UgQiUgg"^) = (7,x,z). Then, by definition ofUgg 

U{go*U^g'^^) = {^ + l,x,z) 

Therefore, 

= (5o*<'))e^(<7o*Z^io)) 
= {go*U^g'J)(B{j + l,x,z) 
= go*{U'^l^ ®{l,x,z)) 
= 90* f 

n)Ug,{U^g;-^^^) = ^. Then 

/ = ® = 
Hence go * f is ^-generated by induction hypothesis. 



As in chapter 4, go < g iff for all x go{x) / =4> goix) = g{x) 
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(2) Let / be a ^p-generated finite zero of Ug. By definition of g, for every m such that, 
for some numbers x,y, 

we have that Z/^(™+^) = hm * O^"^"'^^ for some finite function hm < §■ hm * O*^""^-* is U- 
generated by definition. The sequence of all is increasing, and, by definition of g, the 
limit of all hm is indeed g. Thus, for every finite function h < g we have h < hm < g for 
some finite hm such that hm * O'-'^"^^ is ZY-generated. 

By assumption / is iY^-generated, that is, / = Ug^^ for some n. By continuity oiU{g* f) 

in g, and by continuity of in g for any m < n, we deduce that there is some finite 

function h < g such that for all functions h < go < g the two conditions below hold: 

U{go * f) = U{g * f) 

that is, / is an ZY^^ -generated finite zero of Ug^^. By the discussion above, we may choose 

some finite go such that h < go < g and go * 0^°"^^ is ^/-generated. By point (1), go * f is 
iY-generated: 

go*/ = ^(") (6.2) 

for some n. Suppose 

Uigo*f) = {0,x,z) (6.3) 

for some x, z: we show it is impossible and hence obtain that V({go * f) = 0, by the fact that 
Ugg{f) = and definition oiUg^. Combining (6.2) and (6.3), we obtain 

ZY(Z^W) = (o,x,z) 

By definition of g, for some m < n 

U{U^"''>) = {0,x,y)Ag{x)=y 

This last fact plus (6.3) imply that x ^ x, since by definition U is an update procedure: 
impossible. 

□ 

We are now able to prove the main theorem: update procedures generate terminating 
learning processes. 

Theorem 6.3.6 (Termination of Learning Processes). LetU be an update procedure 
of ordinal a. Then, lA has a finite zero. In particular, there exists A; G N such thatU{U^^^) = 
0. 



Proof. We define, by transfinite induction, a function (7 : a ^ (N — )■ N) as follows. 
Suppose we have defined (/-y : N — )■ N, for every 7 < /3 < a. Define the sequence g^i3 : /3 — >■ 
(N ^ N) of them aU 

5f</3 := 7 e /3 g^ 

Then define 



if 3i. Ug^p{ufl^) = (0,x,y) Ai = min{n | 3z Ug^^{U^%) = {0,x,z)} 
otherwise 
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By axiom of choice and classical logic, for every /3, gf<^ and gjs are well defined. So we can 
let 

9 ■= 9<a 

We first want to show that there exists a (3 such that has a Z/^^^^ -generated finite zero. 
We have two cases: 

(1) a is a successor ordinal. Then, by fact 1, ZYg<a is an update procedure of ordinal 
1, which has a ZYg<ci-generated finite zero (see chapter 4). 

(2) a is a limit ordinal. Then, by continuity of U, there is some /3 < a such that for 
all /5 < (5 < a 

Z^(g)=Z^(<7<5*0(°-^)) 
If U{g) = 0, then by definition 6.3.3 

and we are done. liU{g) = {^,71,171), without loss of generality we can assume we 
have chosen /3 such that 7 < /3. Again by definition 6.3.3 of 

and we are done. 

Let now 

I3q := min{/3 | Ug^^ has a -generated finite zero} 
/3o cannot be a successor, otherwise if we let /3o = /3i + 1 

5</3o = 9<ifSi) * 5/3i 

and hence by fact 1 point (2) 

and by reduction lemma 6.3.5 l^g^^^ would have a Z//^^^^ -generated finite zero. But /3o also 
cannot be a limit ordinal, otherwise by reduction lemma 6.3.4, for some 7 < /3o, Z^g<^ would 
have a -generated finite zero. We conclude that /3o = 0. Since Ug^^ = U, we obtain the 
thesis. □ 

6.4. Spector's System B and Typed Update Procedures of Ordinal uj^ 

Zeros of transfinite update procedures cannot in general be computed in Godel's system 
T: as we will show, already update procedures of ordinal u + k, with k G u, can be used to 
give computational interpretation to Elementary Analysis and hence their zeros can be used 
to compute the functions provably total in Elementary Analysis. We will show however that 
Spector's system B is enough to compute zeros. 

Definition 6.4.1 (Bar Recursion Operator, Spector's System B, Type Level of 
Bar Recursion). In the following, we will work with Spector's system B which is Godel's 
T augmented with constants BRt-^o-, '^r,(T respectively of type 



and 



Ti ^ T2 ^ Tg ^ T4 ^ Bool T 
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with 

Ti = (N ^ cr) ^ N 
T2=a* 
T^ = a* ^ {a ^t) 
Ti = a* 

where a* is a type representing finite sequences of objects of type a. The meaning of BRt-^o- 
is defined by the equation 

mr.YGHs=[^\ , '^^^'<\'\ (6.4) 

[//s(Ax'^BR^,^yGif(s*x)) otherwise ^ ' 

where s*x denotes the finite sequence s followed by s denote the function mapping n to 
5,1, if n < to 0°" otherwise, where Sn is the n-th element of s and |s| is the number of 
elements in s. If cr, r, G, H are determined by the context, we we will just write BR(s) in 
place of mr,aYGHs. 

BRt-^o- is said to be bar recursion of type a. The type level of bar recursion BRt-^h of type 
N (said also type 0), is the type level of the constant BRt-^k, that is, assuming N* = N, 
max(l, typelevel(r)) + 2. 

In order to obtain a strongly normalizing system such that equation 6.4 holds, we have to 
add to system B the following reduction rules (see Berger [13]): 

BRr,aYGHs ^ -^r^aYGHsiYs < \s\) 

^r,aYGHs{'Ii:ne) ^ Gs 

^r,aYGHs{Fa.lse) ^ Hs{Xx''BRr,aYGH{s * x)) 

where < is a term coding the correspondent relation on natural numbers. 



Since we are interested only in computable update procedures, we now fix a system 
for representing them. For the aim of computationally interpreting Elementary Analysis, 
update procedures can be assumed to belong to system T. However, for more powerful 
systems one may need more capable update procedures, so we define them to belong to 
B. Here, we limit ourselves to the ordinal u^, for k E uj, since this ordinal is enough to 
interpret Elementary Analysis and even fragments of Ramified Analysis (see for example. 
Mints et al. [36]) 



Definition 6.4.2 (Representation of Ordinals and Typed Update Procedures 
OF Ordinal uj^). We will represent ordinal numbers of the form oj^, with k G w, by 
exploiting the order isomorphism between u'' and N'^ lexicographically ordered. So, for 
k G Lo, k > 0, we set 

[w°] := I/, [w^] := N^' 

where v is the empty string and 

[a;° ^ (N ^ N)] := N ^ N 

and, if /c € 

[^fc+i ^ (N ^ N)] := N ^ [w'^ ^ (N ^ N)] 
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where N is the type representing N in typed lambda calculus. Define moreover 

[{u'' X N X N) U {0}] := [a;''] x N x N 

Unfortunately, does not have a code. So we have to use an injective coding |_| of the 
set (cj'' x N X N) U {0} into the set of closed normal terms of type [{uj'' x N x N) U {0}]. 
To fix ideas, we define n,m)| = (/3',n + l,m + 1), with /3' : n'^ the code of /3, and 

|0| = (o,...,o). 

A typed update procedure of ordinal oj^ is a term of Spector's system B of type: 

[^k ^ (pj ^ i^)] ^ [(^fc X N X N) U {0}] 

satisfying point (2) of definition 6.2.1, where for simplicity function quantification is assumed 
to range over functions definable in system B. Equality as it appears in the definition is 
supposed to be extensional. 

6.5. Bar Recursion Proof of the Zero Theorem for Typed Update Procedures 

of Ordinal o;'^ 

In this section we give a constructive proof of the Zero theorem for typed update pro- 
cedures of ordinal less than u;^. In particular we show that finite zeros can be computed 
with bar recursion of type 1. We start with the base case. 

Theorem 6.5.1 (Zero Theorem for Update Procedures of Ordinal l=a;°). Let 
U be a typed update procedure of ordinal 1. Then hi has a finite zero a. Moreover, a can 
be calculated as the normal form of a bar recursive term Zero(^) (defined uniformly on the 
parameter U) of system T plus bar recursion of type 0. 

The result follows by Oliva [39]. We give below another proof, which is a simplification 
of Oliva's one, made possible by the slightly stronger condition we have imposed on the 
notion of update procedure. 

The informal idea of the construction - but with some missing justifications - is the following. 
We reason over the well-founded tree of finite sequences of numbers s such that U{s) = 
\{n,m)\ and n > \s\. We want to construct a function cr : N — > N which is a zero of U. 
Suppose that we have constructed a "good" initial approximation a{0) * • • • * a{i) of a; we 
want to prove that it can be extended to a long enough approximation of a. Our first step 
is to continue with (t(0) * • • • * a{i) * 0. If this is a good guess, by well-founded induction 
hypothesis, we can extend a{0) * • • • * a{i) * to a complete approximation a{0) * • • • * cr{n) 
of a, with n > i. Since we are not sure that our previous guess was lucky, we compute 
U{a{Qi) * • • • * 0"(n)). If for all m 

U{cr{ld) * ■ ■ ■ * a{n)) / \{i + l,m)\ 

then our approximation for a{i + 1) is adequate, and we claim that (t(0) * • • • * o"(n) is the 
approximation of a we were seeking. Otherwise 

U{a{Q) * ■ ■ ■ * a{n)) = \{i + l,m)\ 

for some m: lA tells us that our guess for the value of -|- 1) is wrong. But now we 
know that cj(0) * • • • * a{i) * m is a good initial approximation of a and we have made 
progress. Again by well-founded induction hypothesis, we conclude that we can extend 
(t(0) * • • • * o{i) * m to a good approximation of a. 
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Proof of Theorem 6.5.1. We formalize and complete the previous informal argument. In 
the following s will be a variable for finite sequences of numbers. Using bar recursion of 
type 0, we can define a term which builds directly the finite zero we are looking for and is 
such that: 



BR(s) 



s if Z//s = |(n, m)| and n < |sj 

s ifZ^s=|0| 

BR(s*m) if Z^(BR(s*0)) = |(|s|,m)| 

_BR(s*0) if Z^(BR(s*0)) / |(|s|,m)| for ah m 

(we assume that BR(s) checks in order every condition in its definition and executes the 
action corresponding to the first satisfied condition). We let a as the normal form of 

Zero(Z^) := BR(()) 

where () is the empty sequence. Let us prove that a is a finite zero of U. Suppose Ua 
\{n,m)\: by showing that this is impossible, we obtain that Ua = 
BR(()) leads to the following chain of equations: 

BR(0) = BR(a(0)) 

= BR(o-(0) *cr(l)) 



The normalization of 



= BR(cr(0) * • • • *a{i)) 

= (t(0) * • • • * a{i) 
= a 

with 

n < \a{0) * • • • * cr{i)\ = i + 1 

In particular 

BR(()) = BR(ct(0) * • • • * a(n - 1)) 

Now, we have two cases: 

(1) U{BR{a{0) * ••• *f7(n - 1) *0)) = |(n,0|. Then 

BR(()) = BR(cj(0) *---*a{n-l)*l) 

and so o"(n) = /, which is impossible, by definition 6.2.1 of update procedure, point 
(2), for Ua = \{n,m)\. 

(2) for alU, ^/(BR(ct(0) * • • • *cr(n - 1) *0)) / |(n,/|). Then by definition 

BR(o-(0) *a{n-l)) = BR(o-(0) * ■ ■ ■ * a{n - 1) * 0) 
Therefore 

\{7i,m)\ = Ua = U{BR{{))) = U{BR{a{0) * ■ ■ ■ * a{n - 1) * 0)) 
again impossible, by assumption of this case. 
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We have then proved that a is the sought finite zero. 

□ 

We now prove that every typed update procedure of ordinal u has a finite zero. 

Theorem 6.5.2 (Zero Theorem for Typed Update Procedures of Ordinal uj). 
Let U he a typed update procedure of ordinal uj. Then lA has a finite zero a. Moreover, a 
can be calculated as the normal form of a bar recursive term ZerOuj{<U) (defined uniformly 
on the parameter U) of system T plus bar recursion of type 1 := N ^ N. 



Proof. The finite function a : — > (N — > N)] we are going to construct can be 
represented as a finite function sequence cr(0) * cj(1) * • • • * cr(n), for a large enough n. In the 
following s is a variable ranging over finite sequences of natural number functions. Using 
bar recursion of type 1, we can define in a most simple way a term which builds directly 
the finite zero we are looking for. We present the construction gradually. To begin with, 
suppose we are able to define - uniformly on s - terms BR(s) and : (N — >■ N) satisfying the 
following equation for every s: 

{§ if Z^/s = 1(7, n, m)| and 7 < |s| 

s ifZ^s=|0| 
BR(s*5's) otherwise, where V?i, m U{BR{s * gs)) {\s\,n,m) 

Let 

a := Zero^(Z^) := BR(()) 

We prove that u is a finite zero of U. We show this by proving that Ua = (7, n, m) is 
impossible. As in the proof of theorem 6.5.1 

BR(()) = BR(cr(0) * • • • * cr(i)) = cr(0) a{i) 

with 7 < i + 1. Let 

r := (j(0) * • • • * cj(7 — 1) 

By some computation 

Ua=U{BR{{))) 

= Z^(BR(cj(0) * • • • * cr(7 - 1))) 
= Z^(BR(r)) 
= U{BR{r*gr)) 
Since by construction for all n, m 

U{BR{r*gr)) ^ |(|r|,n,m)| = |(7,n,m)| 

we obtain that lAa 7^ {"f^n,m): impossible. 

It remains to show that a gs such that appears in the definition of BR(s) exists. Indeed, it 
is enough to set 

gs := Zero(Ar^"W,,|(BR(s*/))) 
where, for z G N, we have defined 

Ui := xf^i^^^), \fU{f) = \{i,n,m)\ then |(n,m)| else |0| 
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We prove now that in fact U{BR(s * gs)) 7^ |(|s|,n,m)| for all n,m. First, observe again 
that for every s 

BR(s) = s * hi * ■ ■ ■ * hn 

for some terms /ii, . . . , /i„ of type N — )• N. Now, fix any finite sequence s of type-N N terms. 
We want to show that 

Fs:=Xf^''U\4BR{s*f)) 

is an update procedure of ordinal 1. Suppose F^gi = \{n,m)\, g2{n) = m and Fsg2 = \ {h^l)\. 
Then, by definition of Fs^ it must be that 

U{BR{s*gi)) = 

and 

U{BR{s*g2)) = \{\sW)\ 

Moreover, 

BR(s * g2)\s\{n) = g2{n) = m 

Since U is an update procedure, h ^ n must hold; therefore Fs is an update procedure of 
ordinal 1. But by definition of gs, Zero and theorem 6.5.1, this means that 

|0| =F,(Zero(F,)) =^^|,|(BR(s*9,)) 

By definition of it must be true that U{BR{s * gs)) 7^ Id's!, f^-)! for all n,m. 

□ 

The previous argument can be generalized in order to prove the Zero theorem for typed 
update procedures of ordinal o;'^. 



Theorem 6.5.3 (Zero Theorem for Typed Update Procedures of Ordinal u'^, 
WITH k ^ uj). Let U be a typed update procedure of ordinal . Then lA has a finite zero 
a. Moreover, a can be calculated as the normal form of a bar recursive term Zero^k{U) 
(defined uniformly on the parameter U) of system T plus bar recursion of some type A, 
where typelevel(^) = 1. 



Proof. By induction on k. The cases k = 0,1 have already been taken care. Now, we 
want to prove the thesis for k+1, with A; > 0. The finite function a : [u^^^ ^ (N — > N)] we 
are going to construct can be represented as a finite function sequence a{0)*a{l) *■ ■ ■ *a{n), 
for a large enough n, with each a{i) of type [w'^ — > (N ^ N)]. In the following s represents 



a sequence of functions of type [uj (N 



We recall that an ordinal less than 0;*^+^ 



is coded as a pair (7,/3), with 7 E N and /3 € N'^. We again present the construction 
gradually. To begin with, suppose we are able to define - uniformly on s - terms BR(s) and 
gs '■ [uj^ — >■ (N — 7> N)] satisfying the following equation for every s: 

|((7,/3),n,m)| and 7 < \s\ 



BR(s) 



if Z^s 
if Us 



Let 



BR(s*(7s) otherwise, where V/3, n, m ^(BR(s * (7^)) / ((|s|, /3), n, m) 



a := Zero^k+i(^/) := BR(()) 
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We prove that a is a finite zero of U. We show this by proving that Ua = ((7, /?), n, m) is 
impossible. As in the proof of theorem 6.5.1 

BR(()) = BR(cr(0) * • • • * a{i)) = cr(0) cj{i) 

with 7 < i + 1. Let 

r := o"(0) * • • • * cj(7 — 1) 

By some computation 

Ua=U{BR{{))) 

= U{BR{a{0) *---*a(7-l))) 
= ^(BR(r)) 
= Z^(BR(r*5,)) 

Since by construction 

Z^(BR(r * <7r)) / |((|r|,/3),n,m)| = |((7, n, m)| 
we obtain that Ua 7^ ((7, n, m): impossible. 

It remains to show that a Qs such that appears in the definition of BR(s) exists. Indeed, it 
is enough to set 

gs := Zero,.(A/[-''-MW|,|(BR(. * /))) 
where, for i G N, we have defined 

Ui := ^jt-^^+^^CN^N)]^ If ^^y.) ^ (^^i^S),n,m) then {6,n,m) else 

We prove now that in fact U{BR{s * gs)) 7^ |((|s|, /3), n, ?ti)| for all f3,n,m. First, observe 
that for every s 

BR(s) = s * hi * ■ ■ ■ * hn 

for some terms /ii, . . . , /i„ of type N — )• N. Now, fix any finite sequence s of type-N N terms. 
First, we want to show that 

Fs:= A/[-'^M^|(BR(s*/)) 

is an update procedure of ordinal 00^. Suppose for some 6 of type [co^]: Fsgi = \{6,n,m)\, 
y6o < 6. {gi)so = {g2)so, {92)sin) = m and Fsg2 = \i6,h,l)\. Then, by definition of F^, it 
must be that 

U{BRis*gi)) = \{{\s\,5),n,m)\ 

and 

U{BR{s*g2)) = ms\,6),h,l)\ 

Since U is an update procedure and 

BR(s * g2)\s\5{n) = {g2)5{n) = m 

then h ^ n must hold; therefore Fg is an update procedure of ordinal co^. But by definition 
of gs and induction hypothesis, this means that 

^,|(BR(s * gs)) = Fs{Zero^,{Fs)) = |0| 

By definition oiU^s\ it must be true that U{BR{s * gs)) 7^ for all /3,?i, m. 

□ 
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6.6. Case Study: Elementary Analysis 

In this section, we give a three-step description of the epsilon substitution method for 
Elementary Analysis. Every step corresponds to one of the three stages in which - according 
to section 6.2 - learning based computational interpretations of predicative classical Arith- 
metic can be decomposed. As main foundational result one obtains a constructive proof that 
the zero theorem for update procedures of ordinal less that oj ■ 2 implies the 1-consistency 
of Elementary Analysis. More precisely, any zero of such an update procedure can be used 
to compute witnesses for IIq formulas. 

The content of this section is based on Mints et al. [35] and may be considered as an 
informal survey and a general guide to the reading of the epsilon substitution method in 
the light of our ideas on learning. Neither full details nor full proofs will be provided, but 
our description should be clear enough for the reader to gain an understanding of the basic 
ideas underpinning the epsilon method and its learning based interpretation. 

We first define the language of Elementary Analysis, which is a fragment of second order 
Arithmetic in which second order quantification ranges over arithmetical formulas (possibly 
with free set variables). 

Definition 6.6.1 (Language >CeaOF EA). The terms of £ea are inductively defined as 
follow: 

(1) Numerical variables x,y,z, . . . are terms of type 0. 

(2) Set variables X,Y,Z,... are terms of type 1. 

(3) is a term of type 0. 

(4) If t is a term of type 0, S(t) is term of type 0. 
The formulas of Cea are inductively defined as follows: 

(1) For every natural number n, there is a denumerable set of n-ary predicate con- 
stants, one for every computable predicate over n-uples of natural numbers. If P 
is a Tx-ary predicate constant and t^^, . . . are terms of type 0, then Pti . . .tn is 
an atomic formula. 

(2) If t is a term of type and X a variable of type 1, then t € A is an atomic formula. 

(3) If A and B are formulas, then A A B, A ^ B, -^A are formulas. 

(4) If A is a formula and is a variable, 3vA is a formula and \/vA is defined as -^3v^A. 

If A is a formula and z a variable of type free in A, then \zA is a lambda set] XzA is 
said to be arithmetical if it contains no bound set variables. The formula B(XzA/X) is 
defined as the formula obtained from B by substituting each atomic formula t € A of S 
with A{t/z), as usual without capture of variables. 

We now define the axioms and inference rules of EA. 
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Definition 6.6.2 (Axioms and Inference Rules of EA). The axioms of EA are for- 
mulas of £ea defined as follows: 

(1) Propositional tautologies are axioms. 

(2) Definitions of predicate constants are axioms, e.g. for add predicate 

add{x, 0, x) and add{x, y, z) — > add{x, S(y), S(z)) 

(3) X = X and x = y A{x) — )> A{y) are equality axioms. 

(4) -'S(x) = and S(x) = S(y) —^x = y are axioms. 

(5) ^(0) (yx.A{x) —7- ^(5(3;))) yxA{x) is the induction axiom scheme. 

(6) A{t/x) — )> 3xA is an axiom for every term t of type 0. 

(7) A{T/X) — )■ 3XA is an axiom if T is a set variable or an arithmetical lambda set. 
The inference rules of EA are modus ponens 

A^ B A 
B 

and 

A^C 
3vA C 

with the standard proviso that v does not occur free in C . 

We are now ready to take the first step of a learning based interpretation. 

6.6.1. First Stage: Identification of a Sequence of non Computable Functions 

F. We now define the sequence of non computable functions needed to give a computational 
interpretation of EA. We do that by first introducing the concept of epsilon term. 

Definition 6.6.3 (Language /^eAe)- We define by simultaneous induction the terms and 
the formulas of C^^e- 



(1) 


Numerical variables x,y,z, . . . are terms of type 0. 




(2) 


Set variables X,Y, Z, . . . are terms of type 1. 




(3) 


is a term of type 0. 




(4) 


If i is a term of type 0, S(t) is term of type 0. 




(5) 


For every n-ary predicate constant P of £eA) if ^i, • • • , in are 
Pti . . .tn is an atomic formula. 


terms of type 0, then 


(6) 


If i is a term of type and T a term of type 1, then t € T is 


an atomic formula. 
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(7) If ^ and B are formulas, then A A B, A ^ B, ^A are formulas. 

(8) If A is a formula and w is a variable, then evA is an epsilon term (of type equal to 
the type of v) and v is considered bound in evA. 

If ^ is a formula and z a variable of type free in A, then \zA is a lambda term (but we 
do not ask it is in C^At)- A formula or a lambda term or a term is said to be an expression 
of EAe and is arithmetical if it contains no bound set variables and canonical if it is closed 
(i.e. no free variables occurs in it) and does not have closed epsilon terms as subterms. XzA 
is regular if it is of the form {\zB)[ti/vi . . . tn/vn] with XzB arithmetical and ti, . . . , t„ any 
terms. 

Canonical epsilon terms are the ones that are assigned a meaning. The intended deno- 
tation of a canonical epsilon term exA is the least number n such that A{n) is true while 
the denotation of eXA is an arithmetical canonical lambda term XzG (which represents 
an arithmetical set) such that A{\zG/X) is true. In other words, the following critical 
formulas should be true: 

A{t/x) A{exA/x) 

A{T/X) AieXAjX) 

where t is any term of type and T is an epsilon term of type 1 or a regular lambda term. 
The notion of truth for formulas of EAe requires further explanation: in order to evaluate 
their truth, first epsilon terms must be evaluated and hence eliminated. 

Definition 6.6.4 (Substitutions, Evaluations of Epsilon Terms). We define: 

(1) An epsilon substitution 5 is a function from the set of canonical epsilon terms to 
the set of numerals and arithmetical canonical lambda terms such that S{exA) is 
always a numeral and S{eXA) is always an arithmetical canonical lambda term. 

(2) Let ti,t2 be expressions of EAe and S an epsilon substitution. We write ti 1—^5 t2 if 
t2 is obtained from ti either by substituting one of its canonical epsilon subterms 
ex A with S{exA) or replacing one of its subformulas t € eXA with G{t), where 
S{eXA) = XzG. 

(3) An expression t of EAe is said to be in S-normal form if there is no ti such that 
t i-^s ti- We indicate with \t\s the unique 5-normal form of t, which exists by 
theorem 6.6.1 below. 

The truth value of a closed formula A of EAe is the truth value of 1^415' (which does not 
contain epsilon terms): so it is always relative to a epsilon substitution S. 

The relation >-^s is well founded and has Church- Rosser property (see Mints et al. [35]). 

Theorem 6.6.1 (Normalization and Church-Rosser). For every S, the relation i-^s 
is well founded and every expression t of EAe has an S -normal form. 
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We now need to measure the "computational strength" of an expression t of EAe. In- 
tuitively, from the computational point of view, an epsilon term exA represent a recursion 
theoretic jump, because in general one has to enumerate all natural numbers in order to 
decide if an n exists such that |A(n)|5 is true. So, closed arithmetical expressions will 
have a computational strength below uj, because no more than a finite number of jumps 
is done inside them by their epsilon subterms. Instead, an epsilon term eXA must have a 
computational strength of at least u because one has to know all the values of arithmetical 
canonical epsilon terms in the first place if he wants to determine whether there exists a 
canonical arithmetical lambda term XzG such that \A(XzG/X)\s is true. Indeed one can 
assign to expressions a computational strength which is always less that co ■ 2. This is done 
through the so called rank function, which we introduce only by exposing the properties 
that it must have (for the actual definition and details, see Mints et al. [35]). 

Theorem 6.6.2 (Rank Function). There exists a function rk from the set of expressions 
of EAe to UJ ■ 2 such that the following holds. For every epsilon substitution S and ordinal 
a, denote with S<:a the function mapping e to S{e) if rk(e) < a, to or 0^ := (Az .z = z) 
otherwise (according to the type of e); then 

(1) For every canonical epsilon term evA 

rk{evA) > rk{A{e/v)) 
whenever e is a numeral or an arithmetical canonical lambda term. 

(2) For every expression e of EAe and epsilon substitutions 81,82, if rk(e) = a and 
{8i)<a = {82)<a, then \e\si = \e\s2- 

The above theorem 6.6.2 is crucial and its meaning is the following. Given any canonical 
epsilon term ex A, any substitution 8 and natural number n = 8 {ex A), in order to check 
whether n is a correct denotation for exA, we have to determine the truth value of 

\A{n/x)\s 

Since rk{exA) is strictly greater than rk{A{n/x)), the truth of |^(n/x)|5 depends only on 
the values that 8 assigns to epsilon terms of rank strictly less than that of ex A. So the 
meaning of exA is predicatively determined by the meaning of epsilon terms of lower rank. 
The same holds for canonical epsilon terms of type 1. 

We are now able to define the sequence of functions that will enable us to define classical 
witness for formulas in EA and that we shall try to approximate. Suppose that for every 
ordinal a < 2 ■ uj we have a primitive recursive enumeration eQ , e^ , . . . of canonical epsilon 
terms of rank equal to a and a primitive recursive enumeration Aq, Ai, . . . , of canonical 
arithmetical lambda terms. We associate to any function / : w • 2 ^ (N ^ N) the epsilon 
substitution 8f such that: 

„ , , \m if 8 f(e'^) = m A e'^ = exA 
^ ' \l \i8f{eZ) = kf\eZ = eXA 

It is easy to see - using classical logic - that there exists an epsilon substitution S which 
makes true every critical formula C, i.e. \C\s is true: just start by assigning values to 
canonical epsilon terms of rank 1, then to those of rank 2 and so on. Then our target 
collection F : w • 2 — (N — )■ N) of functions is the one such that S = 8f. 
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6.6.2. Second Stage: Definition of Classical Witnesses by Programs Recur- 
sive in F. Using epsilon terms one can define classical witness for any provable formula of 
EA by first translating formulas of EA into formulas of EAe, using tlie equivalence 

3vA = A[evA/v) 

Definition 6.6.5 (Translation of Formulas of EA into Formulas EAe). We define 
a translation of Formulas of EA into Formulas EAe by induction as follows: 

(1) If P is atomic, P* := P. 

(2) {^A)* := -nA*. 

(3) (AAB)* := A* AB*. 

(4) {A B)* := A* B*. 

(5) {3vA)* := A*{evA*/v) 

We now define the axioms and inference rules for EAe. 

Definition 6.6.6 (Axioms and Inference Rules of EAe). The axioms of EAe are 
formulas of C^Ae defined as follows: 

(1) Propositional tautologies are axioms. 

(2) Definitions of predicate constants are axioms, e.g. for add predicate 

add{x, 0, x) and add{x, y, z) — > add{x, S(y), S(z)) 

(3) X = X and x = y — > A{x) — > A{y) are equality axioms. 

(4) -'S(x) = and S{x) = S{y) —^x = y are axioms. 

(5) Minimality axioms: exA = S{t) — )> ^A{t) 

(6) Critical formulas: 

-is = ^ s = S(exs = 5(2;)) 
A{t/x) A{exA/x) 
A{T/X) A{eXA/X) 

where t is any term of type and T is either a term of type 1 or a regular lambda 
term. 

The only inference rule of EAe is modus ponens. 

The following theorem shows that EA can be embedded in the quantifier free system 
EAe. This allows one to extract witnesses for existential statements provable in EA. As 
usual h denotes provability. 
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Theorem 6.6.3 (Classical Witnesses for Provable Formulas of EA). The fol- 
lowing holds: 

(1) Suppose that EA h A. Then EAe h A*. 

(2) Suppose that EA h 3xA, with A atomic. Then there exists a finite sequence 
Ci , . . . , Cn of closed critical formulas of EAe and a closed term t of EAe such 
that if for all i, \Ci\s is true, then \t\s = n and A(n) is true. 

By the above theorem 6.6.3, it is now clear that witnesses for EA can be computed by 
programs recursive in F, because F represent an epsilon substitution Sp which makes every 
critical formula true. 

6.6.3. Stage Three: Learning Processes Approximating F. In order to compute 
witnesses for EA is necessary to find a good finite approximation of F, in order to satisfy 
some finite set of critical formulas. This is the point when update procedures come into the 
scene. The fundamental property of a closed critical formula C, for example of the form 

A{t/x) A{€xA/x) 

is that from the fact \C\s is false one can always learn something. Suppose \C\s is false. 
In this case, if S'(ea;|A|5') = m, one has that |74(m/x)|5 is false. Fortunately, if \t\s = n, 
the formula [^(n/x)!^ is true since \C\s is false and so |A(no/x)|5 is true for some minimal 
no < n. So one learns a new value uq that can be assigned to exjAj^. Analogously, if C is 
a closed critical formula of the form 

A{T/X) A{eXA/X) 

and \C\s is false, if we suppose S'(eX|74|5') = XzG, one has that \A{XzG/X)\s is false. 
Fortunately, if |r|5 = XzH, the formula \ A{XzH/X)\s happens to be true. So one learns a 
new value XzH that can be assigned to eXl^j^. Observe that is not a priori obvious that 
XzH is an arithmetical lambda term, but in fact this can be proved given the assumptions 
we have made on T (again, for details see Mints et al. [35]). 

From now on, fix a finite sequence of critical formulas Co, ... , Cn (for brevity only of the 
two forms considered above). We want to define an update procedure of ordinal oj + k out 
of it (with k uj), any of whose finite zeros will represent an epsilon substitution making 
all the critical formulas true. 

Definition 6.6.7 (Update Procedure for Cq, . . . , Cn) ■ We define an update procedure 
l/l of ordinal u + k. Let / : a; + A: — )■ (N — )■ N). If for every i, \Ci\sj: is true, we set 

m) = 

Otherwise, consider the first i such that |Cj|s^ is false. If 

d = A{t/x) A{exA/x) 
and \t\sj = m and ex|A|5^. = e", we set 

^(/) = {a,n,mo) 

where mo < m is the smallest among the i such that \A{i/x)\sj: is true (which exists since 
\A{m/x)\sf is true). If 

d = A{T/X) A{eXA/X) 
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and \T\sf = XzH = Am and eX|A|5^ = e", we set 

^(/) = {a,n,m) 

We now show that U is a well defined update procedure. 

Theorem 6.6.4 (Adequacy of U). U is an update procedure of ordinal u + k, for some 
k G CO. 

Proof. We skip the proof that U is well defined, which amounts to show that for every 
substitution S, if rk(e) > w, then rk(|e|5) < rk(e): this ensures an upper bound on the rank 
of the terms that are evaluated by U in its computations and so U never updates values for 
epsilon terms of rank greater than u + k, k is chosen large enough. 

We prove instead that U is an update procedure. U is continuous, since for every / only a 
finite number of values of Sf and hence of / are used to compute lA{f). Suppose now that 
U{f) = {/3,n,m), for all 7 < /3 = g^, gpi^) = rn and U{g) = {f3,h,l). Suppose h = n: 
we have to prove it is impossible. Consider the first i such that |Ci|5^ is false. Suppose Cj 
is of the form 

d = A{t/x) A{exA/x) 

Then by definition of U{f), ex\A\sj: = and m is the smallest among the i such that 
\A{i/x)\sj: is true. Furthermore, consider the first j such that jCjl^g is false. By definition 
of h{{g) and since h = n we have 

Cj = B{t/v) B{evB/v) 
with ev\B\sg = Therefore, = l^lsg- Moreover, let 

8:= A{\B\sg{m/x)) 
By theorem 6.6.2, point (1), 6 < /3 = rk{ex\B\sg)- By hypothesis 

{Sg)<5 = {Sf)<S 

So by theorem 6.6.2, point (2) 

\A{m/x)\sf = \\A\sfim/x)\sf = \\B\sg{m/x)\sj = \\B\sgim/x)\sg = \B{m/x)\sg 
Since 5/3 (n) = m, we have 

Sgiev\B\sg) = Sgief^) = m 

Thus 

\BievB/v)\s, = \Bim/x)\s, 
But \A{m/x)\sj. is true by construction and so \B{evB /v)\sg itself must be true, which 
contradicts the assumption that jCjl^g is false. 

An analogous reasoning yields a contradiction when Cj is of the form 

A(T/x) ^ A{eXA/X) 

□ 

Theorem 6.6.1 (1-Consistency of Elementary Analysis). If for all k e ui every 
update procedure of ordinal uj + k has a finite zero, then Elementary Analysis is 1-consistent. 
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Proof. By theorem 6.6.3, it is enough to show that, given any finite sequence of critical 
formulas (Ci, . . . , Cat without loss of generality), there exists a finite epsilon substitution 
that makes true every formula. This amounts to show that lA has a finite zero, which is 
true by theorem 6.6.4 and hypothesis. □ 

6.7. Further Work 

Much remains to be done and we plan to address the following issues in the future. 

Our constructive proof of the zero theorem for typed update procedures of ordinal uj^ 
is not optimal, in the sense that, by Howard [30], it should be possible to use only system 
T plus bar recursion of type 0. 

Moreover, a more self contained proof that the zero theorem for update procedures of 
ordinal less that oo ■ 2 implies the consistency of EA is a major aim. 
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